PPL Electric Utilities - PA.Gov · 2019. 11. 26. · PPL Electric Utilities Annual Report to the Pennsylvania Public Utility Commission PHASE III OF ACT 129 PY10 ANNUAL REPORT (JUNE

PPL Electric Utilities Annual Report to the Pennsylvania Public Utility Commission

PHASE III OF ACT 129 PY10 ANNUAL REPORT

(JUNE 1, 2018 – MAY 31, 2019)

FOR PENNSYLVANIA ACT 129 OF 2008

ENERGY EFFICIENCY AND CONSERVATION PLAN

Prepared by:

Cadmus

Prepared for:

PPL Electric Utilities

November 15, 2019

Table of Contents PPL Electric Utilities | i

Table of Contents

1 Introduction .......................................................................................................................... 1

1.1 Executive Summary ....................................................................................................................... 5

2 Summary of Achievements ..................................................................................................... 8

2.1 Carryover Savings from Phase II of Act 129 .................................................................................. 8

2.2 Phase III Energy Efficiency Achievements to Date ........................................................................ 8

2.3 Phase III Demand Response Achievements to Date ................................................................... 11

2.4 Phase III Performance by Customer Sector ................................................................................ 13

2.5 Summary of Participation by Program ........................................................................................ 14

2.6 Summary of Impact Evaluation Results ...................................................................................... 15

2.7 Summary of Energy Impacts by Program .................................................................................... 16

2.8 Summary of Demand Impacts by Program ................................................................................. 20

2.9 Summary of Fuel Switching Impacts ........................................................................................... 23

2.10 Summary of Cost-Effectiveness Results ...................................................................................... 24

2.11 Comparison of Performance to Approved EE&C Plan ................................................................ 27

2.12 Summary of Process Evaluation Results ..................................................................................... 31

2.13 Findings and Recommendations ................................................................................................. 35

3 Evaluation Results by Program ............................................................................................. 36

3.1 Impact Evaluation ....................................................................................................................... 36

3.2 Process Evaluation ...................................................................................................................... 37

4 Portfolio Finances and Cost Recovery ................................................................................... 39

4.1 Program Finances ........................................................................................................................ 39

4.2 Cost Recovery .............................................................................................................................. 40

5 Non-Residential Energy Efficiency Program ........................................................................... 42

5.1 Gross Impact Evaluation ............................................................................................................. 43

5.2 Net Impact Evaluation ................................................................................................................. 44

5.3 Verified Savings Estimates .......................................................................................................... 45

5.4 Cost-Effectiveness Reporting ...................................................................................................... 45

6 Non-Residential Efficient Equipment Program ...................................................................... 48

6.1 Participation and Reported Savings by Customer Segment – Lighting ....................................... 49

6.2 Gross Impact Evaluation – Lighting ............................................................................................. 50

Table of Contents PPL Electric Utilities | ii

6.3 Net Impact Evaluation – Lighting ................................................................................................ 51

6.4 Verified Savings Estimates – Lighting .......................................................................................... 52

6.5 Participation and Reported Savings by Customer Segment – Equipment .................................. 53

6.6 Gross Impact Evaluation – Equipment ........................................................................................ 54

6.7 Net Impact Evaluation – Equipment ........................................................................................... 55

6.8 Verified Savings Estimates – Equipment ..................................................................................... 56

6.9 Process Evaluation – Lighting and Equipment ............................................................................ 56


6.11 Recommendations – Lighting and Equipment ............................................................................ 63

7 Non-Residential Midstream Lighting Program ....................................................................... 65

7.1 Participation and Reported Savings by Customer Segment ....................................................... 66






7.7 Recommendations ...................................................................................................................... 78

8 Non-Residential Custom Program ......................................................................................... 79







8.7 Recommendations ...................................................................................................................... 92

9 Non-Residential Continuous Energy Improvement Program .................................................. 93






9.6 Cost-Effectiveness Reporting .................................................................................................... 101

Table of Contents PPL Electric Utilities | iii

9.7 Recommendations .................................................................................................................... 102

10 Efficient Lighting Program .................................................................................................. 103

10.1 Participation and Reported Savings by Customer Segment ..................................................... 104

10.2 Gross Impact Evaluation ........................................................................................................... 105

10.3 Net Impact Evaluation ............................................................................................................... 109

10.4 Verified Savings Estimates ........................................................................................................ 110

10.5 Process Evaluation .................................................................................................................... 110



11 Home Energy Education ..................................................................................................... 125

11.1 Participation and Reported Savings .......................................................................................... 127



11.4 Uplift Analysis ........................................................................................................................... 131





12 Winter Relief Assistance Program ....................................................................................... 151








13 Energy Efficient Home Program .......................................................................................... 171






Table of Contents PPL Electric Utilities | iv



14 Appliance Recycling Program ............................................................................................. 192








15 Energy Efficiency Kits and Education Program ..................................................................... 207








16 Student Energy Efficient Education Program ....................................................................... 224








17 Demand Response Program ............................................................................................... 237






Table of Contents PPL Electric Utilities | v

Upstream Lighting Cross-Sector Sales .......................................................................... A-1

Site Inspection Summary............................................................................................. B-1

Home Energy Report Impact Evaluation Detail .............................................................. C-1

Evaluation Detail – Efficient Equipment Program ......................................................... D-1

Evaluation Detail – Midstream Lighting Program ........................................................... E-1

Evaluation Detail – Custom Program ............................................................................. F-1

Evaluation Detail – Continuous Energy Improvement Program .................................... G-1

Evaluation Detail – Efficient Lighting Program ............................................................. H-1

Evaluation Detail – Home Energy Education Program ...................................................... I-1

Evaluation Detail – Winter Relief Assistance Program .................................................... J-1

Evaluation Detail – Energy Efficient Home Program ......................................................K-1

Evaluation Detail – Appliance Recycling Program .......................................................... L-1

Evaluation Detail – Energy Efficiency Kits and Education Program .............................. M-1

Evaluation Detail – Student Energy Efficient Education Program .................................. N-1

Evaluation Detail – Demand Response Program .......................................................... O-1

Non-Energy Benefits .................................................................................................... P-1

Net Impact Evaluation ................................................................................................ Q-1

Survey Methodology ................................................................................................... R-1

Table of Contents PPL Electric Utilities | vi

Figures

Figure 1-1. PY10 Verified Savings by Sector .................................................................................................. 6

Figure 2-1. EE&C Plan Performance Toward Phase III Portfolio Target ........................................................ 9

Figure 2-2. EE&C Plan Performance Toward Phase III Low-Income Compliance Target ............................ 10

Figure 2-3. EE&C Plan Performance Against Phase III GNE Compliance Target ......................................... 11

Figure 2-4. Event Performance Compared to 85% Per-Event Target ......................................................... 13

Figure 2-5. PY10 PYTD Energy Savings by Energy Efficiency Programs....................................................... 17

Figure 2-6. P3TD Energy Savings by Energy Efficiency Programs ............................................................... 17

Figure 2-7. PYTD Demand Savings by Energy Efficiency Program .............................................................. 21

Figure 2-8. P3TD Demand Savings by Energy Efficiency Program .............................................................. 21

Figure 2-9. Portfolio-Level Program Satisfaction Very and Somewhat Satisfied Combined ....................... 33

Figure 2-10. PY10 Program Satisfaction by Sector Very and Somewhat Satisfied Combined .................... 34

Figure 2-11. PY10 Program Satisfaction by Individual Program Very and Somewhat Satisfied

Combined ....................................................................................................................................... 35

Figure 6-1. PY10 Participant Satisfaction .................................................................................................... 58

Figure 6-2. Participant Satisfaction with Different Program Elements ....................................................... 60

Figure 6-3. PY10 Areas Working Well in the Efficient Equipment Program ............................................... 61

Figure 6-4. Participants’ Average Influence Scores..................................................................................... 62

Figure 7-1. PY9 and PY10 Distributor, Contractor, and End-User Satisfaction ........................................... 75

Figure 8-1. PY10 Overall Custom Program Satisfaction .............................................................................. 87

Figure 8-2. PY10 Participant Satisfaction with Custom Program Components .......................................... 88

Figure 8-3. PY10 Custom Program Areas that Are Working Well ............................................................... 89

Figure 8-4. PY10 Reasons Participants Completed Custom Program Energy Efficiency Project ................ 89

Figure 8-5. PY10 Average Influence Score for Different Items in the Custom Program ............................. 90

Figure 8-6. PY10 Most Important Criteria for Completing a Custom Program Project .............................. 90

Figure 10-1. Light Bulb Purchase Patterns, by Technology – Residential ................................................. 115

Figure 10-2. Light Bulb Purchase Patterns, by Source – Small Business ................................................... 116

Figure 10-3. PY10 Satisfaction with LEDs, by Sector ................................................................................. 117

Figure 10-4. PY10 Satisfaction with LED Traits – Residential .................................................................... 117

Figure 10-5. PY10 Satisfaction with LED Traits – Small Business .............................................................. 118

Figure 10-6. Importance of LED Traits to Purchase Decision, by Program Year – Residential ................. 119

Figure 10-7. Other LED Traits Important to Purchase Decision – Residential .......................................... 120

Figure 10-8. Importance of LED Traits to Purchase Decision by Program Year – Small Business ............. 120

Figure 11-1. Overall Satisfaction with Home Energy Reports ................................................................... 140

Figure 11-2. Overall Satisfaction with Home Energy Reports by Wave .................................................... 141

Figure 11-3. PY10 Readership of Home Energy Reports ........................................................................... 142

Figure 11-4. Agreement Level to Statements about the Home Energy Reports ...................................... 142

Figure 11-5. Familiarity with Energy Efficiency Programs ........................................................................ 143

Figure 11-6. Self-Reported Visits to Utility Website and Energy Management Portal ............................. 144

Figure 11-7. Self-Reported Adoption of Energy-Saving Practices ............................................................. 145

Figure 12-1. Participant Satisfaction with WRAP ...................................................................................... 163

Table of Contents PPL Electric Utilities | vii

Figure 12-2. Participant Satisfaction with Different Program Components ............................................. 164

Figure 13-1. Overall Program Satisfaction for PY10, PY9, and PY8 ........................................................... 183

Figure 13-2. PY10 Program Feature Satisfaction: All Program Delivery Elements ................................... 184

Figure 13-3. Areas Working Well: Equipment, Weatherization, and In-Home Audit Components ......... 185

Figure 13-4. Areas Working Well: Online Assessment Component .......................................................... 185

Figure 14-1. Proportion of Equipment Recycled in PY10 (Percentage of Total Units) ............................. 200

Figure 14-2. Participant Overall Satisfaction ............................................................................................ 202

Figure 14-3. PY10 Participant Satisfaction With Program Components ................................................... 202

Figure 14-4. Program Components That Worked Best or Next Best for Participants .............................. 203

Figure 15-1. Participant Satisfaction with Energy Efficiency Kit ............................................................... 218

Figure 16-1. Participant Satisfaction with Student Energy Efficient Education Program Overall by

Cohort .......................................................................................................................................... 234

Figure 17-1. Gross Verified Savings in Comparison to Act 129 Targets .................................................... 242

Figure 17-2. Satisfaction with Timing of Event Notifications and Online Event Enrollment Process ....... 249

Figure 17-3. Overall Satisfaction with Demand Response Program ......................................................... 250

Figure 17-4. Satisfaction with the Incentive Amount ............................................................................... 251

Figure 17-5. Overall Satisfaction with the ICSP ......................................................................................... 251

Figure C-1. Percentage Daily Savings across Time: Legacy Wave 1 ......................................................... C-11

Figure C-2. Percentage Daily Savings across Time: Legacy Wave 2 ......................................................... C-11

Figure C-3. Percentage Daily Savings across Time: Expansion Wave 1 .................................................... C-12

Figure C-4. Percentage Daily Savings across Time: Low-Income Wave 1 ................................................ C-12

Figure C-5. Percentage Daily Savings across Time: Low-Income Wave 2 ................................................ C-13

Figure C-6. Percentage Daily Savings across Time: Phase III Expansion Wave 1 ..................................... C-13

Figure C-7. Percentage Average Daily Savings from First Month of Treatment ...................................... C-14

Figure E-1. Midstream Lighting Program PY10 Reported Energy Savings by Customer Segment ............. E-1

Figure E-2. Midstream Lighting Program PY10 Reported Product Savings by Customer Segment ........... E-2

Figure E-3. Influences on Contractor’s and Client’s Efficient Lighting Decisions ..................................... E-12

Figure G-1. Optimizing HDD and CDD Base Temperatures for School G .................................................. G-3

Figure L-1. Secondary Market Impacts—Refrigerators ............................................................................. L-7

Figure L-2. Refrigerator NTG Combined Decision Tree ............................................................................ L-10

Figure L-3. Freezer NTG Combined Decision Tree ................................................................................... L-11

Figure N-1. Ex Post Verified Savings Methodology Flow Diagram ............................................................ N-1

Figure O-1. PPL Electric Utilities Act 129 Gross Verified Demand Savings, PY10 ..................................... O-7

Table of Contents PPL Electric Utilities | viii

Tables

Table 2-1. PY10 Energy Efficiency Achievements to Date ............................................................................ 8

Table 2-2. Phase III Energy Efficiency Achievements to Date ....................................................................... 9

Table 2-3. PY10 Demand Response PYVTD Performance by Event ............................................................ 12

Table 2-4. PY10 Summary Statistics by Customer Sector ........................................................................... 13

Table 2-5. Phase III Summary Statistics by Customer Sector ...................................................................... 14

Table 2-6. EE&C Portfolio Participation by Program ................................................................................... 14

Table 2-7. Impact Evaluation Results Summary .......................................................................................... 15

Table 2-8. High-Impact Measure Net-to-Gross ........................................................................................... 16

Table 2-9. Incremental Annual Energy Savings by Program (MWh/Year) .................................................. 18

Table 2-10. Savings by Reported and Verified Year .................................................................................... 19

Table 2-11. Lifetime Energy Savings ........................................................................................................... 19

Table 2-12. Peak Demand Savings by Energy Efficiency Program (MW/Year) ........................................... 22

Table 2-13. Verified Gross Demand Response Impacts by Program .......................................................... 23

Table 2-14. Phase III Fuel Switching Summary ........................................................................................... 23

Table 2-15. Summary of Portfolio Finances – Gross Verified ..................................................................... 24

Table 2-16. PY10 Gross TRC Ratios by Program ($1,000) ........................................................................... 25

Table 2-17. PY10 Net TRC Ratios by Program ($1,000) ............................................................................... 26

Table 2-18. P3TD Gross TRC Ratios by Program ($1,000) ........................................................................... 26

Table 2-19. P3TD Net TRC Ratios by Program ($1,000) .............................................................................. 27

Table 2-20. Comparison of PY10 Expenditures to Phase III EE&C Plan ($1,000) ........................................ 28

Table 2-21. Comparison of P3TD Expenditures to Phase III EE&C Plan ($1,000) ........................................ 28

Table 2-22. Comparison of PY10 Actual Program Savings to EE&C Plan Projections for PY10 ................... 29

Table 2-23. Comparison of Phase III Actual Program Savings to EE&C Plan Projections for Phase

III .................................................................................................................................................... 29

Table 2-24. PY10 Participant Surveys and Program Satisfaction Response Counts ................................... 32

Table 3-1. PY10 Evaluation Activity Matrix ................................................................................................. 36

Table 3-2. PY10 Impact Evaluation Activities by Program .......................................................................... 37

Table 3-3. PY10 Process Evaluation Activities by Program ......................................................................... 38

Table 4-1. PY10 Program and Portfolio Total Finances ............................................................................... 39

Table 4-2. P3TD Program and Portfolio Total Finances .............................................................................. 40

Table 4-3. EE&C Plan Expenditures by Cost-Recovery Category(1) ($1,000) ............................................... 41

Table 5-1. Non-Residential Energy Efficiency Program Savings .................................................................. 43

Table 5-2. PY10 Non-Residential Energy Efficiency Program Participation and Reported Impacts ........... 43

Table 5-3. PY10 Non-Residential Energy Efficiency Program Gross Energy Savings Realization

Rates by Component ...................................................................................................................... 44

Table 5-4. PY10 Non-Residential Energy Efficiency Program Gross Impact Results for Energy ................. 44

Table 5-5. PY10 Non-Residential Energy Efficiency Program Gross Impact Results for Demand ............... 44

Table 5-6. PY10 Non-Residential Energy Efficiency Program NTG Ratios by Component .......................... 45

Table 5-7. PYTD and P3TD Non-Residential Energy Efficiency Program Savings Summary ........................ 45

Table 5-8. Summary of Program Finances – Gross Verified........................................................................ 46

Table of Contents PPL Electric Utilities | ix

Table 5-9. Summary of Program Finances – Net Verified ........................................................................... 47

Table 6-1. PY10 Efficient Equipment Program (Lighting) Participation and Reported Impacts ................. 50

Table 6-2. PY10 Efficient Equipment Program (Lighting) Gross Impact Evaluation Sample Design ........... 50

Table 6-3. Efficient Equipment (Lighting) Program Savings ........................................................................ 50

Table 6-4. PY10 Efficient Equipment Program (Lighting) Gross Impact Results for Energy........................ 51

Table 6-5. PY10 Efficient Equipment Program (Lighting) Gross Impact Results for Demand ..................... 51

Table 6-6. PY10 Efficient Equipment Program (Lighting) Net Impact Evaluation Sample Design .............. 52

Table 6-7. PY10 Efficient Equipment Program (Lighting) Net Impact Evaluation Results .......................... 52

Table 6-8. PYTD and P3TD Efficient Equipment Program (Lighting) Savings Summary .............................. 53

Table 6-9. PY10 Efficient Equipment Program (Equipment) Participation and Reported Impacts ............ 53

Table 6-10. PY10 Efficient Equipment Program (Equipment) Gross Impact Evaluation Sample

Design............................................................................................................................................. 54

Table 6-11. Efficient Equipment (Equipment) Program Savings ................................................................. 54

Table 6-12. PY10 Efficient Equipment Program (Equipment) Gross Impact Results for Energy ................ 54

Table 6-13. PY10 Efficient Equipment Program (Equipment) Gross Impact Results for Demand .............. 55

Table 6-14. PY10 Efficient Equipment Program (Equipment) Net Impact Evaluation Sample

Design............................................................................................................................................. 55

Table 6-15. PY10 Efficient Equipment Program (Equipment) Net Impact Evaluation Results ................... 56

Table 6-16. PYTD and P3TD Efficient Equipment Program (Equipment) Savings Summary ....................... 56

Table 6-17. Process Evaluation Sampling Strategy for the Efficient Equipment Program .......................... 57

Table 6-18. Comments and Suggested Improvements for the Efficient Equipment Program

Overall ............................................................................................................................................ 59

Table 6-19. Suggested Improvements for Elements of the Efficient Equipment Program ......................... 60

Table 6-20. Status of Recommendations for the Efficient Equipment Program ........................................ 64

Table 7-1. PY10 Midstream Lighting Program Participation and Reported Impacts .................................. 66

Table 7-2. PY10 Midstream Lighting Program Gross Impact Evaluation Sample Size ................................ 67

Table 7-3. PY10 Midstream Lighting Impact Evaluation Activities ............................................................. 68

Table 7-4.Midstream Lighting Program PY8 through PY10 Participation Growth ...................................... 68

Table 7-5. PY10 Midstream Lighting Program Gross Impact Results for Energy ........................................ 68

Table 7-6. PY10 Midstream Lighting Program Gross Impact Results for Demand ..................................... 69

Table 7-7. PYTD and P3TD Midstream Lighting Program Savings Summary .............................................. 70

Table 7-8. Process Evaluation Sampling Strategy for Midstream Lighting ................................................. 72

Table 7-9. Percentage of Commercial Rebate Customers Aware of Midstream Lighting Program ........... 74

Table 7-10. Midstream Lighting Awareness................................................................................................ 74

Table 7-11. Status of Recommendations for Midstream Lighting .............................................................. 78

Table 8-1. PY10 Custom Program Participation and Reported Impacts ..................................................... 81

Table 8-2. PY10 Program Custom Project Types ......................................................................................... 81

Table 8-3. PY10 Custom Program Gross Impact Evaluation Sample Design ............................................... 82

Table 8-4. Custom Program Savings ........................................................................................................... 82

Table 8-5. PY10 Custom Program Gross Impact Results for Energy ........................................................... 83

Table 8-6. PY10 Custom Program Gross Impact Results for Demand ........................................................ 83

Table 8-7. PY10 Custom Program Net Impact Evaluation Sample Design .................................................. 84

Table of Contents PPL Electric Utilities | x

Table 8-8. PY10 Custom Program Net Impact Evaluation Results .............................................................. 85

Table 8-9. PY10 Custom Program Free Ridership Comparison by Stratum ................................................ 85

Table 8-10. PYTD and P3TD Custom Program Savings Summary ............................................................... 85

Table 8-11. PY10 Custom Program Process Evaluation Sampling Strategy ................................................ 86

Table 8-12. Status of Recommendations for the Custom Program ............................................................ 92

Table 9-1. Participating Schools in Each District in the Continuous Energy Improvement Program ......... 95

Table 9-2. PY10 Continuous Energy Improvement Program Participation and Reported Impacts ............ 95

Table 9-3. Continuous Energy Improvement Program Savings .................................................................. 95

Table 9-4. Continuous Energy Improvement Program Gross Impact Results for Energy ........................... 96

Table 9-5. Continuous Energy Improvement Program Gross Impact Results for Demand ........................ 96

Table 9-6. Continuous Energy Improvement Net Impact Evaluation Sample Design................................. 97

Table 9-7. PYTD and P3TD Continuous Energy Improvement Program Savings Summary ........................ 97

Table 9-8. PY10 Continuous Energy Improvement Component Process Evaluation Sampling

Strategy .......................................................................................................................................... 98

Table 9-9. Status of Recommendations for Continuous Energy Improvement Program ......................... 102

Table 10-1. PY10 Efficient Lighting Program Participation and Reported Impacts................................... 105

Table 10-2. PY10 Efficient Lighting Program Gross Impact Evaluation Sample Design ............................ 106

Table 10-3. PY10 Efficient Lighting Verified Program Savings .................................................................. 106

Table 10-4. PY10 Efficient Lighting Program Gross Impact Results for Energy......................................... 106

Table 10-5. PY10 Efficient Lighting Program Gross Impact Results for Demand ...................................... 106

Table 10-6. PY10 Efficient Lighting Program Manufacturer Invoice Sample Sizes ................................... 107

Table 10-7. PYTD and P3TD Efficient Lighting Program Savings Summary ............................................... 110

Table 10-8. Efficient Lighting Program Process Evaluation Sampling Strategy ........................................ 111

Table 10-9. Program Unit Sales by Retail Channel and Program Year ...................................................... 112

Table 10-10. Program Sales by Bulb Type and Program Year ................................................................... 112

Table 10-11. Residential General Population Recent LED Purchasers and Non-Purchasers .................... 113

Table 10-12. Summary of Efficient Lighting Program Finances – Gross Verified ...................................... 122

Table 10-13. Summary of Efficient Lighting Program Finances – Net Verified ......................................... 123

Table 10-14. Status of Recommendations for Efficient Lighting .............................................................. 124

Table 11-1. PY10 Home Energy Education Program Participation and Reported Impacts ....................... 128

Table 11-2. PY10 Home Energy Education Program Gross Impact Sample Design .................................. 129

Table 11-3. Home Energy Education Program Savings ............................................................................. 129

Table 11-4. PY10 Home Energy Education Program Gross Impact Results for Energy ............................. 130

Table 11-5. Home Energy Education Program Gross Impact Results for Demand ................................... 130

Table 11-6. PY10 Home Energy Education Participation Uplift Summary ................................................ 132

Table 11-7. PY10 Home Energy Education Downstream Uplift Energy Savings Summary ....................... 133

Table 11-8. PY10 Home Energy Education Downstream Uplift Demand Savings Summary .................... 134

Table 11-9. PY10 Home Energy Education Upstream Lighting Uplift Energy Savings Summary .............. 134

Table 11-10. PY10 Home Energy Education Upstream Lighting Uplift Demand Savings Summary ......... 135

Table 11-11. PYTD and P3TD Home Energy Education Program Savings Summary ................................. 136

Table 11-12. Process Evaluation Sampling Strategy ................................................................................. 137

Table 11-13. Home Energy Report Delivery Frequency ............................................................................ 139

Table of Contents PPL Electric Utilities | xi

Table 11-14. Significant Behavioral Differences between ........................................................................ 146

Table 11-15. Summary of Home Energy Education Program Finances–Gross and Net Verified .............. 147

Table 11-16. Status of Recommendations for the Home Energy Education Program ............................. 150

Table 12-1. WRAP Participation and Reported Impacts ........................................................................... 154

Table 12-2. PY10 WRAP Gross Impact Evaluation Sample Design ............................................................ 156

Table 12-3. Winter Relief Assistance Program Savings ............................................................................. 156

Table 12-4. PY10 WRAP Gross Impact Results for Energy ........................................................................ 157

Table 12-5. PY10 WRAP Gross Impact Results for Demand ...................................................................... 157

Table 12-6. PYTD and P3TD WRAP Savings Summary ............................................................................... 158

Table 12-7. WRAP Process Evaluation Sampling Strategy ........................................................................ 160

Table 12-8. Recalled Recommended Behaviors ........................................................................................ 165

Table 12-9. Summary of WRAP Finances–Gross and Net Verified ........................................................... 167

Table 12-10. Status of Recommendations for WRAP ............................................................................... 169

Table 13-1. Energy Efficient Home Program Participation and Reported Impacts .................................. 174

Table 13-2. PY10 Energy Efficient Home Program Gross Impact Sample Design ..................................... 175

Table 13-3. Energy Efficient Home Program Savings ................................................................................ 176

Table 13-4. Energy Efficient Home Program Gross Impact Results for Energy......................................... 176

Table 13-5. Energy Efficient Home Program Gross Impact Results for Demand ...................................... 176

Table 13-6. Energy Efficient Home Program – Efficient Equipment Stratum Net Impact

Evaluation Results ........................................................................................................................ 179

Table 13-7. Energy Efficient Home Program Net Impact Evaluation Results ........................................... 179

Table 13-8. PYTD and P3TD Energy Efficient Home Program Savings Summary ...................................... 180

Table 13-9. Process Evaluation Sampling Strategy ................................................................................... 181

Table 13-10. PY10 Satisfaction: Program Delivery Elements by Component (Percent Satisfied or

Very Satisfied) .............................................................................................................................. 184

Table 13-11. Summary of Energy Efficient Home Program Finances–Gross Verified .............................. 186

Table 13-12. Summary of Energy Efficient Home Program Finances–Net Verified .................................. 187

Table 13-13. Status of Recommendations for the Energy Efficient Home Program ................................ 191

Table 14-1. Eligible Equipment and Incentives for the Appliance Recycling Program ............................. 193

Table 14-2. PY10 Appliance Recycling Program Participation and Reported Impacts ............................. 194

Table 14-3. PY10 Appliance Recycling Program Gross Impact Evaluation Sample Design ....................... 195

Table 14-4. Appliance Recycling Program Savings ................................................................................... 195

Table 14-5. PY10 Appliance Recycling Program Gross Impact Results for Energy ................................... 195

Table 14-6. PY10 Gross Energy Results by Product Recycled ................................................................... 196

Table 14-7. PY10 Appliance Recycling Program Gross Impact Results for Demand ................................. 196

Table 14-8. Current and Historical Net-to-Gross Ratios ........................................................................... 198

Table 14-9. PYTD and P3TD Appliance Recycling Program Savings Summary .......................................... 198

Table 14-10. PY10 Appliance Recycling Program Process Evaluation Sampling Strategy ........................ 199

Table 14-11. How Participants Learned About the Appliance Recycling Program ................................... 201

Table 14-12. Digital Sources for How Participants Learned About the Appliance Recycling

Program ....................................................................................................................................... 201

Table 14-13. Summary of Appliance Recycling Program Finances–Gross Verified .................................. 204

Table of Contents PPL Electric Utilities | xii

Table 14-14. Summary of Appliance Recycling Program Finances–Net Verified ...................................... 205

Table 15-1. Products Included in PY10 Energy Efficiency Kits .................................................................. 209

Table 15-2. Energy Efficiency Kits and Education Program Participation and Reported Impacts ............ 210

Table 15-3. Energy Efficiency Kits and Education Gross Impact Sample Design for PY10 ........................ 211

Table 15-4 Energy Efficiency Kits and Education Program Savings .......................................................... 211

Table 15-5. Energy Efficiency Kits and Education Program Gross Impact Results for Energy .................. 211

Table 15-6. Energy Efficiency Kits and Education Program Gross Impact Results for Demand................ 212

Table 15-7. PY10 Reported vs. Evaluated ISRs .......................................................................................... 213

Table 15-8. PYTD and P3TD Energy Efficiency Kits and Education Program Savings Summary................ 214

Table 15-9. PY10 Energy Efficiency Kits and Education Program Process Evaluation Sampling

Strategy ........................................................................................................................................ 215

Table 15-10. Summary of Energy Efficiency Kits and Education Program Finances–Gross and Net

Verified ......................................................................................................................................... 221

Table 15-11. Status of Recommendations Energy Efficient Kits and Education ....................................... 223

Table 16-1. PY10 Student Energy Efficient Education Participation and Reported Impacts .................... 226

Table 16-2. PY10 Student Energy Efficient Education Program Gross Impact Evaluation Sample

Design........................................................................................................................................... 228

Table 16-3. Student Energy Efficient Education Program Savings ............................................................ 228

Table 16-4. PY10 Student Energy Efficient Education Program Gross Impact Results for Energy ........... 228

Table 16-5. PY10 Student Energy Efficient Education Gross Impact Results for Demand ........................ 229

Table 16-6. Verified Student Energy Efficient Education ISRs for Products by Year ................................. 230

Table 16-7. PYTD and P3TD Student Energy Efficiency Education Program Savings Summary ................ 231

Table 16-8. Student Energy Efficient Education Program Process Evaluation Sampling Strategy ........... 232

Table 16-9. Summary of Student Energy Efficient Education Program Finances–Gross and Net

Verified ......................................................................................................................................... 235

Table 17-1. Dual-Enrolled Participants ..................................................................................................... 239

Table 17-2. PY10 Demand Response Program Participation and Reported Demand Reductions ........... 240

Table 17-3. PY10 Demand Response Program Gross Impact Sample Design ........................................... 241

Table 17-4. PY10 Demand Response Program Gross Impact Results for Demand ................................... 243

Table 17-5. PYTD and P3TD Demand Response Program Savings Summary ............................................ 244

Table 17-6. PY10 Process Evaluation Sampling Strategy .......................................................................... 245

Table 17-7. PY10 Demand Response Participant Survey Sample Attrition Table ..................................... 246

Table 17-8. PY10 Demand Response Enrolled Company and Survey Respondent Profile ....................... 248

Table 17-9. Ease/Difficulty of Participating in PY10 Events ...................................................................... 249

Table 17-10. Summary of Demand Response Program Finances – Gross Verified .................................. 253

Table 17-11. Status of Recommendations for the Demand Response Program ...................................... 255

Table A-1. PY10 Cross-Sector Proportion Calculations by Sector ............................................................. A-2

Table A-2. PY10 Cross-Sector Assumptions by Sector .............................................................................. A-3

Table B-1. Site Inspection Summary .......................................................................................................... B-1

Table C-1. PY10 Customer Attrition ........................................................................................................... C-1

Table C-2. PY10 Sample Attrition from Data Cleaning ............................................................................... C-2

Table of Contents PPL Electric Utilities | xiii

Table C-3. PY10 Tests for Significant Differences in Annual Pretreatment Consumption ......................... C-3

Table C-4. Treatment Effects for the Home Energy Education Program by Model Specifications ............ C-7

Table C-5. Percentage Treatment Effects by Model Specifications ........................................................... C-8

Table C-6. PY9 Home Energy Education Program Savings Estimate ........................................................ C-10

Table C-7. Participation Uplift by Program (Per 1,000 Customers) ......................................................... C-16

Table C-8. Savings Uplift by Program ....................................................................................................... C-18

Table D-1. Efficient Equipment Program (Lighting) Sampling Strategy .................................................... D-2

Table D-2. PY10 Quarterly Efficient Equipment Program (Lighting) Substrata Boundaries ..................... D-2

Table D-3. PY10 Efficient Equipment Program (Lighting) Post-Stratification ........................................... D-3

Table D-4. Efficient Equipment Component (Equipment) Sampling Strategy .......................................... D-5

Table D-5. PY10 Efficient Equipment Program Lighting and Equipment Stratum Sampling

Strategy for Net Savings Research ............................................................................................... D-6

Table D-6. Energy Equipment Program Intention, Influence, and Free Ridership Score by Stratum ....... D-7

Table D-7. PY10 Efficient Equipment Program NTG Ratio Summary ........................................................ D-7

Table D-8. Total and Survey Population by Sector.................................................................................... D-8

Table D-9. Efficient Equipment Program Participant Survey Sample Attrition Table ............................... D-9

Table E-1. PY10 Midstream Lighting Impact Evaluation Stratum Definitions and Sample Sizes ............... E-3

Table E-2. PY10 Midstream Lighting Impact Evaluation Sampling Strategy .............................................. E-4

Table E-3. PY10 Midstream Lighting Verified Savings Adjustments Summary Shown in Order of

Frequency ..................................................................................................................................... E-6

Table E-4. Distributor Sales Estimates Across Three Categories of Lighting Product Efficiency ............. E-10

Table E-5. Contractor Recommendations and Client Requests ............................................................... E-11

Table E-6. Midstream Lighting Program Sample Attrition Table ............................................................. E-13

Table F-1. PY10 Custom Program Intention, Influence, and Free Ridership Scores .................................. F-3

Table F-2. PY10 Custom Program Free Ridership for Four Top Saving Projects ........................................ F-4

Table F-3. PY10 Custom Program Sector Breakdown ................................................................................ F-4

Table F-4. PY10 Facility Types of Custom Program Participants and Survey Respondents ....................... F-5

Table F-5. PY10 Custom Program Participant Survey Sample Attrition Table ........................................... F-6

Table G-1. Evaluated Schools and Districts ............................................................................................... G-1

Table G-2. Independent Variables by Sampling Interval .......................................................................... G-2

Table G-3. Evaluated Energy Savings by School ........................................................................................ G-4

Table G-4. Evaluated Average Demand Savings by School (kW/yr).......................................................... G-5

Table G-5. PY10 Continuous Energy Improvement Sampling Strategy for NTG Research ....................... G-6

Table G-6. PY10 Continuous Energy Improvement NTG Findings ............................................................ G-7

Table H-1. PY10 Baseline Wattage Adjustments by PA TRM Bulb Category ............................................ H-1

Table H-2 and Table H-3 list the total number of records submitted to the survey subcontractor

and the outcome (final disposition) of each record. ................................................................... H-2

Table I-1. Treatment Group Customer Survey Sample Attrition Table ....................................................... I-1

Table I-2. Statistical Weights for Overall Treatment Group Customer Survey Data................................... I-2

Table J-1. PY10 Winter Relief Assistance Program Sampling Strategy ...................................................... J-1

Table J-2. PY10 Baseload Items for Winter Relief Assistance Program ..................................................... J-2

Table J-3. PY10 Low-Cost Items for Winter Relief Assistance Program ..................................................... J-2

Table of Contents PPL Electric Utilities | xiv

Table J-4. PY10 Full-Cost Items for Winter Relief Assistance Program ...................................................... J-3

Table J-5. PY10 WRAP Participant Counts ................................................................................................. J-6

Table J-6. PY10 Record Review Findings .................................................................................................... J-6

Table J-7. Verified Energy Education Savings and Assumptions Summary Table ...................................... J-8

Table J-8. Calculation and Results of Energy Education Percent Change Components ............................ J-8

Table J-9. Comparison of ISRs in Phase III .................................................................................................. J-9

Table J-10. Winter Relief Assistance Program Logic Model Review ........................................................ J-10

Table J-11. PY10 WRAP Sample Attrition Table for Participant Telephone Surveys ............................... J-11

Table K-1. PY10 Reported vs. Evaluated ISRs for In-Home Audit and Online Assessment Kit

Products ........................................................................................................................................ K-6

Table K-2. Satisfaction Detail by Program Component .............................................................................. K-6

Table K-3. Suggested Improvements ......................................................................................................... K-8

Table K-4. Energy Efficient Home Sample Attrition Table ......................................................................... K-9

Table L-1. Historical Part-Use by Appliance Type (1) ................................................................................. L-2

Table L-2. Prospective Part-Use by Appliance Type (1) ............................................................................... L-3

Table L-3. Part-Use Adjusted Gross Per-Unit Savings ................................................................................ L-3

Table L-4. UEC Input Comparison for Refrigerator and Freezer Savings Algorithms ................................. L-4

Table L-5. PY10 Participant Survey Sampling Strategy .............................................................................. L-5

Table L-6. Final Distribution of Kept and Discarded Participant Appliances.............................................. L-6

Table L-7. Final Distribution of Kept and Discarded Participant Appliances.............................................. L-7

Table L-8. Spillover Equipment, Reported Quantities, and Savings Sources ............................................. L-8

Table L-9. Appliance Recycling Program Equipment Spillover by Equipment Category ............................ L-9

Table L-10. PY10 NTG Ratios ...................................................................................................................... L-9

Table L-11. Appliance Recycling Program Online Survey Attrition .......................................................... L-12

Table M-1. Criteria for Assigning Survey Ex Ante and Survey Verified Savings ....................................... M-1

Table M-2. 2016 PA TRM Open Variables ................................................................................................ M-2

Table M-3. Accuracy of PY10 Data for Energy Efficiency Kits and Education Program ........................... M-3

Table M-4. PY10 Energy Efficiency Kits and Education Program Product-Level ISRs .............................. M-4

Table M-5. PY10 Energy Efficiency Kits and Education Program Showerhead ISRs ................................ M-4

Table M-6. Energy Education Savings by Delivery Method and Kit Type ................................................ M-5

Table M-7. Electric Water Heater Temperature Reduction Savings ........................................................ M-6

Table M-8. Washing More Loads of Laundry in Cold Water Savings ....................................................... M-6

Table M-9. Energy Efficiency Kits and Education Program Customer Survey Demographics by

Delivery Channel ......................................................................................................................... M-1

Table N-1. Criteria for Assigning Survey Ex Ante and Survey Verified Savings ......................................... N-2

Table N-2. Example of Assigning Survey Ex Ante and Survey Verified Savings ......................................... N-3

Table N-3. Pennsylvania TRM Algorithm Open Variables for Innovation Pilot Cohort ............................. N-4

Table O-1. PY10 Program Sampling Strategy ............................................................................................ O-1

Table O-2. Data Sources ............................................................................................................................ O-2

Table O-3. Number of Facilities by Baseline Modeling Approach ............................................................ O-5

Table P-1. Non-Residential End-Use Penetration and Fuel Shares ............................................................ P-2

Acronyms xv

Acronyms

BDR Behavioral Demand Response

C&I Commercial and Industrial

CDD Cooling Degree Day

CEI Continuous Energy Improvement

CF Coincidence Factor

CFL Compact Fluorescent Lamp

CHP Combined Heat and Power

C.L. Confidence Limit

CSP Conservation Service Provider or Curtailment Service Provider

Cv Coefficient of Variation

DLC Direct Load Control

DR Demand Response

EDC Electric Distribution Company

EDT Eastern Daylight Time

EE&C Energy Efficiency and Conservation

EM&V Evaluation, Measurement, and Verification

EISA Energy Independence and Security Act

EUL Effective Useful Life

GNE Government, Nonprofit, Educational

HDD Heating Degree Day

HER Home Energy Report

HIM High Impact Measure

HOU Hours of Use

HVAC Heating, Ventilating, and Air Conditioning

ICSP Implementation Conservation Service Provider

IPMVP International Performance Measurement and Verification Protocol

ISR In-Service Rate

kW Kilowatt

kWh Kilowatt-hour

KPI Key Performance Indicator

LED Light-Emitting Diode

LIURP Low-Income Usage Reduction Program

M&V Measurement and Verification

MW Megawatt

MWh Megawatt-hour

NPV Net Present Value

Acronyms xvi

NTG Net-to-Gross

N/A Not Applicable

O&M Operations and Maintenance

P3TD Phase III to Date

PA PUC Pennsylvania Public Utility Commission

PAC Program Administrator Cost

PSA Phase III to Date Preliminary Savings Achieved; equal to VTD + PYRTD

PSA+CO PSA savings plus Carryover from Phase II

PY Program Year: for example, PY8, from June 1, 2016, to May 31, 2017

PYRTD Program Year Reported to Date

PYVTD Program Year Verified to Date

PYTD Program Year to Date

QA/QC Quality Assurance/Quality Control

RTD Phase III to Date Reported Gross Savings

SEER Seasonal Energy Efficiency Rating

SKU Stock Keeping Unit

SWE Statewide Evaluator

T&D Transmission and Distribution

TRC Total Resource Cost

TRM Technical Reference Manual

VTD Phase III to Date Verified Gross Savings

WRAP Weatherization Relief Assistance Program

Types of Savings xvii

Types of Savings

Gross Savings: The change in energy consumption and/or peak demand that results directly from

program-related actions taken by participants in an EE&C program, regardless of why they participated.

Net Savings: The total change in energy consumption and/or peak demand that is attributable to an

EE&C program. Depending on the program delivery model and evaluation methodology, the net savings

estimates may differ from the gross savings estimate due to adjustments for the effects of free riders,

changes in codes and standards, market effects, participant and nonparticipant spillover, and other

causes of changes in energy consumption or demand not directly attributable to the EE&C program.

Reported Gross: Also referred to as ex ante (Latin for “beforehand”) savings. The energy and peak

demand savings values calculated by the electric distribution company (EDC) or its program

Implementation Conservation Service Providers (ICSP) and stored in the program tracking system.

Unverified Reported Gross: The Phase III Evaluation Framework allows EDCs and the evaluation

contractors the flexibility to not evaluate each program every year. If an EE&C program is being

evaluated over a multi-year cycle, the reported savings for a program year where evaluated results are

not available are characterized as unverified reported gross until the impact evaluation is completed and

verified savings can be calculated and reported.

Verified Gross: Also referred to as ex post (Latin for “from something done afterward”) gross savings.

The energy and peak demand savings estimates reported by the independent evaluation contractor

after the gross impact evaluation and associated M&V efforts have been completed.

Verified Net: Also referred to as ex post net savings. The energy and peak demand savings estimates

reported by the independent evaluation contractor after application of the results of the net impact

evaluation. Typically calculated by multiplying the verified gross savings by a net-to-gross (NTG) ratio.

Annual Savings: Energy and demand savings expressed on an annual basis, or the amount of energy

and/or peak demand an EE&C measure or program can be expected to save over the course of a typical

year. Annualized savings are noted as MWh/year or MW/year. The Pennsylvania (PA) Phase III technical

reference manual (TRM), hereafter referenced as the PA TRM, provides algorithms and assumptions to

calculate annual savings, and Act 129 compliance targets for consumption reduction are based on the

sum of the annual savings estimates of installed measures or behavior change.

Lifetime Savings: Energy and demand savings expressed in terms of the total expected savings over the

useful life of the measure. Typically calculated by multiplying the annual savings of a measure by its

effective useful life. The TRC Test uses savings from the full lifetime of a measure to calculate the cost-

effectiveness of EE&C programs.

Program Year Reported to Date (PYRTD): The reported gross energy and peak demand savings achieved

by an EE&C program or portfolio within the current program year. PYTD values for energy efficiency will

always be reported gross savings in a semi-annual or preliminary annual report.

Types of Savings xviii

Program Year Verified to Date (PYVTD): The verified gross energy and peak demand savings achieved

by an EE&C program or portfolio within the current program year as determined by the impact

evaluation findings of the independent evaluation contractor.

Phase III to Date (P3TD): The energy and peak demand savings achieved by an EE&C program or

portfolio within Phase III of Act 129. Reported in several permutations described below.

▪ Phase III to Date Reported (RTD): The sum of the reported gross savings recorded to date in

Phase III of Act 129 for an EE&C program or portfolio.

▪ Phase III to Date Verified (VTD): The sum of the verified gross savings recorded to date in

Phase III of Act 129 for an EE&C program or portfolio, as determined by the impact

evaluation finding of the independent evaluation contractor.

▪ Phase III to Date Preliminary Savings Achieved (PSA): The sum of the verified gross savings

(VTD) from previous program years in Phase III where the impact evaluation is complete

plus the reported gross savings from the current program year (PYTD).

▪ Phase III to Date Preliminary Savings Achieved + Carryover (PSA+CO): The sum of the

verified gross savings from previous program years in Phase III plus the reported gross

savings from the current program year plus any verified gross carryover savings from Phase

II of Act 129. This is the best estimate of an EDC’s progress toward the Phase III compliance

targets.

▪ Phase III to Date Verified + Carryover (VTD + CO): The sum of the verified gross savings

recorded to date in Phase III plus any verified gross carryover savings from Phase II of Act

129.

PPL Electric Utilities offers nine energy efficiency programs to non-residential, residential and income-verified customers.

PORTFOLIO

Phase III expenditures so far

PY10 expenditures

$163,961

$58,664

87%

86%

of projected

of projected

ACTUAL EXPENDITURES ($1,000)

Phase III has so far saved

PY10 saved

1,130,564 MWh/yr

390,399 MWh/yr

78%

115%

of compliance target

of projected

VERIFIED ENERGY SAVINGS

Satisfied with overall programs88%

PROGRAM SATISFACTION

satisfied with Low-Income programs

satisfied with Non-Residential programs

satisfied with Residential programs

91%

95%

83%

A total of 527,936 participants:

PY10 PARTICIPATION

29,026

Residential

Non-Residential

Low-Income

Demand Response60

8,755

490,095

NON-RESIDENTIAL ENERGY EFFICIENCY PROGRAMFour non-residential programs offer financial incentives to customers in a non-residential rate class.


PY10 expenditures

$50,121

$21,642



PY10 saved

490,939 MWh/yr

184,990 MWh/yr101%of projected


88%of projected



75% satisfied with Continuous EnergyImprovement program

96% satisfied with Custom program

91% satisfied with Efficient Equipment program


PY10 PARTICIPATION

61%of projected

92% satisfied with Midstream Lighting program (distributors)

81%of projected

Continuous Energy Improvement18

72 Custom

Efficient Equipment1,032

Midstream Lighting7,633

Residential customers participate in five programs to recycle inefficient appliances, purchase rebated efficient equipment and discounted lighting, receive home energy reports with tips to save energy, and educate students about energy efficiency.

RESIDENTIAL PROGRAMS


PY10 expenditures

$57,434

$16,583



PY10 saved

589,634 MWh/yr





97% satisfied with the ApplianceRecycling Program

87% satisfied with Energy Efficient Home Program

83% satisfied with the Student Energy Efficient Education Program

258,832


Home Energy Education

Energy Efficient Home

Efficient Lighting

Appliance Recycling14,210

19,863

173,525

PY10 PARTICIPATION

Student EnergyEfficient Education23,665

66% satisfied with the Home Energy Education Program

87%of projected

90%of projected

of projectedof projected87%

LOW-INCOME PROGRAM


PY10 expenditures

$27,256

$9,946

91%

80%

of projected

of projected



PY10 saved

66,524 MWh/yr

28,401 MWh/yr

66%

125%

of projected

of projected


Satisfied with overall program95%

92%

97%


satisfied with the Energy Efficiency Kits and Education Program

satisfied with the Winter Relief Assistance Program


Winter Relief Assistance Program

Energy Efficiency Kits and Education Program

13,564

15,462

PY10 PARTICIPATION

Chapter 1 Introduction PPL Electric Utilities | 5

1 Introduction

Pennsylvania Act 129 of 2008, signed on October 15, 2008, mandated energy savings and demand

reduction goals for the largest electric distribution companies (EDCs) in Pennsylvania for Phase I (2008

through 2013). Phase II of Act 129 began in 2013 and concluded in 2016. In late 2015, each EDC filed a

new energy efficiency and conservation (EE&C) plan with the Pennsylvania Public Utility Commission

(PA PUC) detailing the proposed design of its portfolio for Phase III. These plans were updated based on

stakeholder input and subsequently approved by the PA PUC in 2016.

Implementation of Phase III of the Act 129 programs began on June 1, 2016, and runs until May 2021

(five program years—PY8 to PY12). This report documents the progress and effectiveness of the

Phase III EE&C accomplishments for PPL Electric Utilities in the third program year of Phase III, Program

Year 10 (PY10, June 2018–May 2019), as well as the cumulative accomplishments of the Phase III

programs since inception (June 2016–May 2019).

This report details the participation, spending, reported gross savings, verified gross savings, and verified

net savings impacts of the energy efficiency programs in PY10. Compliance with Act 129 savings goals

are ultimately based on verified gross savings. This report also includes estimates of cost-effectiveness

according to the total resource cost (TRC) test.1

PPL Electric Utilities has retained Cadmus as an independent evaluation contractor for Phase III of

Act 129. Cadmus is responsible for the measurement and verification of the savings and calculation of

gross verified and net verified savings.

Cadmus also conducted a process evaluation for selected programs to examine the design,

administration, implementation, and market response to the Act 129 EE&C programs. This report

presents the key findings and recommendations identified by the impact and process evaluations and

documents PPL Electric Utilities’ consideration of recommendations.

1.1 Executive Summary

PPL Electric Utilities has successfully continued to implement the Phase III Act 129 programs in PY10.

Programs are operating effectively and are meeting or surpassing program objectives. Cadmus does not

suggest any major course corrections. Recommendations suggest minor tuning and possible areas of

inquiry in future years.

1 The Pennsylvania TRC Test for Phase I was adopted by PUC order at Docket No. M-2009-2108601 on June 23, 2009 (2009 PA TRC Test Order). The TRC Test Order for Phase I later was refined in the same docket on August 2, 2011 (2011 PA TRC Test Order). The 2013 TRC Order for Phase II of Act 129 was issued on August 30, 2012. The 2016 TRC Test Order for Phase III of Act 129 was adopted by PUC order at Docket No. M-2015-2468992 on June 11, 2015.


Verified energy savings achieved through PY10 exceed those projected for the phase-to-date shown in

PPL Electric Utilities’ EE&C plan.2 Specifically, PPL Electric Utilities exceeded the PY10 cumulative

projected estimate of 969,545 MWh/yr, achieving 1,130,564 MWh/yr in verified savings, or 117% of

projections through PY10.

Savings achieved through PY10 (1,130,564 MWh/yr) contribute 78% to the Phase III overall five-year

compliance target of 1,443,035 MWh/yr. PPL Electric Utilities is on track to meet the Phase III overall

compliance target.

PPL Electric Utilities is also on track to meet the compliance target for the low-income sector and has

exceeded the compliance target for the government, nonprofit, education (GNE) sector. The low-income

savings target is 79,367 MWh/yr of verified gross energy savings. PPL Electric Utilities has achieved 88%

of the Phase III low-income energy-savings target. The Phase III GNE savings target is 50,507 MWh/yr of

verified gross energy savings. PPL Electric Utilities achieved 231% of the GNE compliance target and

began placing GNE projects on a waitlist in January 2018.

Figure 1-1 shows PPL Electric Utilities’ PY10 program year-to-date (PYTD) verified savings by sector.

Figure 1-1. PY10 Verified Savings by Sector

Note: Total residential sector verified MWh/yr has been adjusted to account for Home Energy Education Program savings uplift.

May not sum to total due to rounding.

PPL Electric Utilities delivered programs for 86% of the PY10 cumulative projected budget estimated in

the EE&C Plan, expending $58,663,622. The acquisition cost in PY10 is $0.15 per annual kWh (EDC

expenditures/first year savings). The portfolio-level total cost of conserved energy (TRC costs/net

present value [NPV] lifetime kWh, at generation) is $0.05. The portfolio-level utility cost of conserved

2 PPL Electric Utilities revised Energy Efficiency and Conservation Plan Act 129 Phase III. Docket No. M-2015-2515642. November 2018.


energy (PAC costs/net present value [NPV] lifetime kWh, at generation) is $0.03. The TRC costs include

PPL Electric Utilities’ costs as well as the customers’ costs. The PAC costs only include PPL Electric

Utilities’ costs.

A portfolio is cost-effective when the TRC benefit-cost ratio exceeds 1.0. The PY10 portfolio is cost-

effective, with a portfolio level TRC of 1.67.

Free ridership is low across the PY10 programs where it was estimated. The evaluated net-to-gross

(NTG) ratio, including spillover attributable to some programs, is 0.81. Program offerings do not need

modification to address free ridership.

In Phase III, PPL Electric Utilities established a goal to achieve 80% or greater of very satisfied and

somewhat satisfied customers in each sector.3 Respondents to participant satisfaction surveys across all

sectors showed high levels of satisfaction with the programs. With the combined very satisfied and

somewhat satisfied responses, portfolio satisfaction for PY10 is 88% (n=19,823); a small but statistically

significant decrease from the PY9 result of 89% (n=21,482).4 The low-income (n=1,093) sector achieved

customer satisfaction of 95%, the nonresidential (n=105) sector achieved customer satisfaction of 91%,

the residential (n=18,625) sector achieved satisfaction of 83%. All three sectors exceeded the customer

satisfaction goal of 80%.

3 The customer satisfaction goal is listed in PPL Electric Utilities’ revised EE&C Plan (Docket No. M-2015-2515642) filed November 2018.

4 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to Pennsylvania Public Utility Commission. Prepared by Cadmus. November 15, 2018.

Chapter 2 Summary of Achievements PPL Electric Utilities | 8

2 Summary of Achievements

2.1 Carryover Savings from Phase II of Act 129

The PA PUC’s Phase III Implementation Order allowed EDCs to carry over savings in excess of the overall

(portfolio) Phase II savings compliance target, in excess of the Phase II GNE savings compliance target,

and in excess of the Phase II low-income savings compliance target.5,6 PPL Electric Utilities did not have

any carryover savings for the portfolio, but it did exceed its Phase II compliance targets for GNE and

low-income.

However, in the August 3, 2017, Compliance Order,7 the PA PUC determined that because PPL Electric

Utilities did not obtain Phase II savings in excess of its Phase II consumption reduction requirement,

PPL Electric Utilities was not entitled to any GNE or low-income sector carryover savings into Phase III.

2.2 Phase III Energy Efficiency Achievements to Date

Table 2-1 shows the achievements to date since the beginning of PY10 on June 1, 2018. Table 2-2 shows

the Phase III achievements to date. The Phase III to-date savings represent 78% of the May 31, 2021,

energy-savings compliance target of 1,443,035 MWh/yr, as shown in Figure 2-1.

Table 2-1. PY10 Energy Efficiency Achievements to Date

PYTD Reported Gross Savings (PYRTD)

Verified Savings (PYVTD)(2)

Unverified (PYRTD)

Realization Rate (2)

Energy Savings (MWh/yr)(1) 416,864 390,399 0 94%

Peak Demand Savings (MW/yr)(1)

168.79 167.52 0 99%

(1) Savings may not match other tables or figures due to rounding. (2) The verified savings and realization rates in this table have been adjusted to account for energy-savings uplift (double-counting) in the Home Energy Education Program.

5 Pennsylvania Public Utility Commission. Energy Efficiency and Conservation Program Implementation Order, Docket No. M-2014-2424864 (Phase III Implementation Order). Entered June 11, 2015.

6 Proportionate to those savings achieved by dedicated low-income programs in Phase III.

7 The Order addresses the EDCs’ compliance with the Phase II energy reduction targets and the Petitions for reconsideration of the April 6, 2017, Compliance Order filed by Duquesne, PECO, and PPL Electric Utilities. Pennsylvania Public Utility Commission. Act 129 Phase II Final Compliance Order. Docket No. M-2012-2289411. Adopted August 3, 2017. http://www.puc.pa.gov/filing_resources/issues_laws_regulations/act_129_information/energy_efficiency_and_conservation_ee_c_program.aspx


Table 2-2. Phase III Energy Efficiency Achievements to Date

P3TD Reported

Gross Savings (P3RTD)

Verified Savings (P3VTD)(2)

Unverified (P3RTD)

Realization Rate (2)

Energy Savings (MWh/yr) (1) 1,197,068 1,130,564 0.00 94%

Peak Demand Savings (MW/yr) (1) 326.61 276.81 0.00 85% (1) Savings may not match other tables or figures due to rounding. (2) The verified savings and realization rates in this table have been adjusted to account for energy-savings uplift (double-counting) in the Home Energy Education Program.

Figure 2-1 summarizes PPL Electric Utilities’ progress, verified to date (VTD), toward the Phase III

portfolio compliance target.

Figure 2-1. EE&C Plan Performance Toward Phase III Portfolio Target

The Phase III Implementation Order directed the EDCs to offer conservation measures to the low-

income customer sector based on the proportion of electric sales attributable to low-income

households. For PPL Electric Utilities, the proportionate number of measures targeted is 9.95%.8

PPL Electric Utilities offers a total of 109 EE&C measures (products and equipment) to its residential and

nonresidential customer classes.9 It makes 22 measures available to the low-income customer sector at

8 Pennsylvania Public Utility Commission. Phase III Implementation Order. Docket No. M-2014-242-2424864. June 11, 2015.

9 PPL Electric Utilities. PPL Electric Utilities Energy Efficiency and Conservation Plan Act 129 Phase III. Docket No. M-2015-2515642. November 2018.


no cost to the customer, which is 20% of the total number of measures offered in the EE&C plan and

exceeds the target of 9.95% for the proportionate number of measures.

The PA PUC also established a low-income energy savings target of 5.5% of the portfolio savings.10 As a

result, the Phase III low-income savings compliance target for PPL Electric Utilities is 79,367 MWh/yr of

verified gross energy savings. Figure 2-2 compares the VTD performance for the low-income customer

sector to the Phase III savings target. Considering verified savings through PY10, PPL Electric Utilities has

achieved 88% of the Phase III low-income energy-savings compliance target.

Figure 2-2. EE&C Plan Performance Toward Phase III Low-Income Compliance Target

The Winter Relief Assistance Program (WRAP) includes savings for multifamily projects that are allocated

to the GNE and small commercial and industrial (C&I) sectors based on the rate class of the buildings’

meters. All savings from this component of the program are counted toward the low-income compliance

target, as set forth in PPL Electric Utilities EE&C Plan. Therefore, the total savings shown in Figure 2-2 do

not match the totals in Table 2-5 below. The additional savings counted toward the low-income

compliance target total 2,657 MWh/yr total: 2,215 MWh/yr from GNE and 442 MWh/yr from small C&I.

10 Pennsylvania Public Utility Commission. Phase III Implementation Order. Docket No. M-2014-242-2424864. June 11, 2015


The Phase III Implementation Order established a GNE energy savings compliance target of 3.5% of the

portfolio savings.11 For PPL Electric Utilities, the GNE compliance target is 50,507 MWh/yr of verified

gross energy savings. Figure 2-3 compares the VTD performance for the GNE customer sector to the

Phase III GNE savings target. Of verified savings through PY10, PPL Electric Utilities has achieved 231% of

the Phase III GNE energy savings compliance target.

Figure 2-3. EE&C Plan Performance Against Phase III GNE Compliance Target

WRAP includes savings for multifamily projects that are allocated to the GNE and small C&I sectors

based on the rate class of the buildings’ meters. All savings from WRAP are counted toward the low-

income compliance target, as set forth in PPL Electric Utilities EE&C Plan. Therefore, the VTD savings in

Figure 2-3 do not include the 1,028 MWh/yr of GNE savings allocated to WRAP and do not match the

GNE savings in Table 2-5 below.

2.3 Phase III Demand Response Achievements to Date

PPL Electric Utilities’ Phase III demand response compliance target is 92 MW/year. Compliance targets

for demand response programs are based on average performance across events and were established

at the system level, which means the load reductions measured at the customer meter must be

escalated to reflect transmission and distribution line losses.

11 Pennsylvania Public Utility Commission. Phase III Implementation Order. Docket No. M-2014-242-2424864. June 11, 2015.


Act 129 demand response events are triggered by PJM Interconnection LLC regional transmission

organization (PJM RTO) day-ahead load forecast. When the day-ahead forecast was above 96% of the

peak load forecast for the year, a demand response event was initiated for the following day. In PY10,

there were six demand response events called.

Table 2-3 lists the dates of the demand response events along with the verified gross demand

reductions achieved by each sector. The table also lists the average demand response performance for

PY10 and for Phase III to date. PPL Electric Utilities’ average demand response performance to date

exceeds the Phase III compliance reduction target of 92 MW by 27%.

Table 2-3. PY10 Demand Response PYVTD Performance by Event

Event Date Start

Hour End Hour

Load Curtailment Portfolio

MW/Event

Impact (1) Small C&I Large C&I GNE

July 2, 2018 2:00 p.m. 6:00 p.m. 1.9 97.2 6.8 105.9

July 3, 2018 2:00 p.m. 6:00 p.m. 1.4 101.8 6.3 109.5

August 6, 2018 2:00 p.m. 6:00 p.m. 1.8 108.1 6.3 116.2

August 28, 2018 2:00 p.m. 6:00 p.m. 1.6 114.5 4.1 120.2

September 4, 2018 2:00 p.m. 6:00 p.m. 1.9 110.9 1.8 114.6

September 5, 2018 2:00 p.m. 6:00 p.m. 1.8 99.2 1.6 102.6

PYVTD - Average PY10 DR Event Performance 111.5

VTD - Average Phase III DR Event Performance 116.6

(1) Portfolio MW/event may not equal sum of customer segment MW/event because of rounding.

The PA PUC’s Phase III Implementation Order also established a requirement that EDCs achieve at least

85% of the Phase III compliance reduction target in each demand response event. For PPL Electric

Utilities, this translates to a 78.2 MW minimum for each demand response event. Figure 2-4 compares

the performance of each of the demand response events in PY10 to the event-specific minimum and

average targets.


Figure 2-4. Event Performance Compared to 85% Per-Event Target

2.4 Phase III Performance by Customer Sector

Table 2-4 presents the participation, savings, and spending by customer sector for PY10. The residential,

Small C&I, and Large C&I sectors are defined by EDC tariff and the residential low-income and GNE

sector were defined by statute (66 Pa. C.S. § 2806.1). The residential low-income sector is a subset of

the residential customer class, and the GNE sector includes customers within the residential, small C&I,

and large C&I rate classes. The savings, spending, and participation values for the low-income and GNE

sectors have been removed from the parent sectors in Table 2-4.

Table 2-4. PY10 Summary Statistics by Customer Sector

Parameter Residential Low-Income Small C&I Large C&I GNE Total (2)

Reported Number of Participants (1) 480,312 29,340 15,278 971 2,413 528,314

PY10 Energy Realization Rate (3) 99% 95% 86% 97% 100% 95%

PYVTD MWh/yr (3) 166,782 30,119 89,978 63,019 46,720 396,617

PY10 Demand Realization Rate (3) 98% 112% 90% 98% 141% 100%

PYVTD MW/yr (Energy Efficiency) (3) 23.14 3.05 14.85 8.15 7.74 56.93

PYVTD MW/yr (Demand Response) - - 1.74 105.28 4.48 111.50

PY10 Incentives ($1000) (4) $6,610 $0 $8,214 $5,665 $2,599 $23,087 (1) Verified participation totals discussed in each chapter and shown in the infographics may differ from the reported participation in this table.

(2) Total may not sum due to rounding. (3) The residential verified savings and realization rates have not been adjusted to account for energy savings uplift (double-counting) in the Home Energy Education Program. (4) The cost of measures provided to low-income participants at no cost is treated as an administrative cost, not as an incentive cost.

Table 2-5 summarizes plan performance by sector since the beginning of Phase III.


Table 2-5. Phase III Summary Statistics by Customer Sector

Parameter Residential Low-Income Small C&I Large C&I GNE Total(1)

Reported Number of Participants 1,222,275 69,424 52,860 1,693 4,844 1,351,096

P3TD Energy Realization Rate (2) 99% 91% 89% 98% 98% 96%

VTD MWh/yr (2) 512,122 67,002 259,611 189,399 118,963 1,147,098

P3TD Demand Realization Rate (2) 59% 108% 90% 103% 116% 91%

VTD MW/yr (Energy Efficiency) (2) 70.07 6.83 42.48 24.48 18.06 161.93

VTD MW/yr (Demand Response) - - 1.7 110.6 4.5 116.6

P3TD Incentives ($1000) (3) $27,581 $0 $16,256 $13,146 $7,140 $64,123 (1) Total may not match sum of columns due to rounding. (2) The residential verified savings and realization rates have not been adjusted to account for energy savings uplift (double-counting) in the Home Energy Education Program. (3) The cost of measures provided to low-income participants at no cost is treated as an administrative cost, not as an incentive cost.

2.5 Summary of Participation by Program

Participation is defined differently for certain programs depending on the program delivery channel and

data tracking practices. These distinctions are summarized by program in Table 2-6, which also provides

the reported participation totals for PY10 and Phase III. PPL Electric Utilities’ tracking database assigns

unique job identifiers to rebated projects, and these correspond to participants as noted in this table.

Verified participation totals discussed in each chapter and shown in the infographics may differ from the

reported participation in this table.

Table 2-6. EE&C Portfolio Participation by Program

Program Participant Definition PYTD Participation P3TD Participation

Appliance Recycling Unique job number; corresponds with each unique appliance decommissioned through the program during the program year.

14,210 38,430

Demand Response Unique account number; corresponds to a customer who enrolled in the program, not the number who participated in at least one event.

64 157

Efficient Lighting Person or business purchasing discounted bulbs. See Efficient Lighting Chapter, section 10.1.1 Definition of a Participant, which describes the approach to computing number of participants.

258,832 883,976

Energy-Efficiency Kits and Education

Unique job number; corresponds to an energy-savings kit delivered to an income-eligible customer through the agency or the direct-mail delivery channel. Participation is determined by the unique job numbers. Returned kits are assigned two unique job numbers: one for the distributed kit, and one for the returned kit.

13,932 39,455

Energy Efficient Home Unique job number; corresponds to a rebated project Households could have more than one rebated project.

19,866 64,601

Home Energy Education Unique bill account number (household) that receives a home energy report in any program year (a household is counted once, even if it received reports in more than one year).

173,525 205,750

Non-Residential Energy Efficiency

Custom: Unique job number; commercially operable job that received an incentive payment during the reporting period.

Continuous Energy Improvement: Unique job number; corresponds to each Individual school.

8,758 16,512


Program Participant Definition PYTD Participation P3TD Participation

Midstream Program: Unique job number (RBT); corresponds to each purchase of discounted products.

Prescriptive Lighting and Equipment: Unique job number; corresponds to each unique job that received a rebate.

Student Energy Efficient Education

Number of participants is counted as the number of energy conservation kits delivered.

23,665 72,024

WRAP

Unique bill account number; corresponds to an income-eligible household that receives an audit and program services.

In PY8, a participant was defined as a unique job number, but the PY9 updated definition is applied retroactively here. Therefore, the P3TD total will not match the PYTD totals from the annual reports.

In PY10, an LED giveaway component was added to the program. The participant count for this component is equal to the number of packs of bulbs given away, 2,450 in PY10.

15,462 30,191

Portfolio Total 528,314 1,351,096

2.6 Summary of Impact Evaluation Results

During PY10, Cadmus completed impact evaluations for all of the energy efficiency programs in the

portfolio, and a net savings analysis for some. Table 2-7 summarizes the realization rates and NTG ratios

by program.

Table 2-7. Impact Evaluation Results Summary

Program Energy

Realization Rate Demand

Realization Rate Net-to-Gross

Ratio

Percentage of Total Portfolio Verified Gross

Verified MWh/yr

Verified MW/yr

Appliance Recycling 79% 85% 0.66 (1) 3% 1%

Demand Response N/A 100% 1.0 (2) 0% 66%

Efficient Lighting 96% 91% 0.83 (3) 27% 8%

Energy Efficiency Kits and Education 77% 116% 1.0 (2) 2% 1%

Energy Efficient Home 86% 87% 0.66 (4) 4% 2%

Home Energy Education 102% 99% 1.0 (5) 11% 4%

Non-Residential Energy Efficiency 97% 106% 0.74 (6) 47% 16%

Student Energy Efficient Education 99% 105% 1.0 (2) 2% 0%

WRAP 90% 93% 1.0 (2) 5% 1%

Total 95% 100% 0.81 (8) 100% (7) 100% (7) (1) PY10 evaluated NTG ratio. (2) No free ridership is expected, nor measured, per the evaluation plan. Therefore, the NTG ratio is 1.0.

(3) PY8 evaluated NTG ratio.

(4) PY9 evaluated NTG ratios used for refrigerator and dehumidifier measures. PY8 evaluated NTG ratios used for all other measures. The 0.66 NTG ratio for the overall program is the verified gross population energy savings weighted average of the NTG ratios applied to each measure. (5) Savings are determined using a randomized control trial and the NTG ratio is irrelevant. (6) PY10 evaluated NTG ratio used for Custom and Efficient Equipment components. PY9 evaluated NTG ratio used for Midstream Lighting. (7) Total may not match sum of rows due to rounding.

(8) Weighted by PY10 program verified gross energy savings.


Findings from net savings research are not used to adjust compliance savings in Pennsylvania. Instead,

this research provides directional information for program planning purposes. Table 2-8 presents

findings for PY10 high-impact measures.

Table 2-8. High-Impact Measure Net-to-Gross

High-Impact Measure Free Ridership Spillover Net-to-Gross Ratio

Efficient Equipment Commercial Lighting (1) 23% (2) 0% 0.77

Custom (1) 35% (2) 0% 0.65

Combined Heating and Power (CHP) (3) N/A N/A N/A

Total 27%(4) 0% 0.73 (1) Estimated from PY10 survey data. (2) Weighted by the survey sample-verified program kWh/yr savings. (3) No CHP participants completed a survey in PY10 despite repeated attempts and outreach from PPL Electric Utilities’ key account manager. CHP projects are included in the Custom program. (4) Weighted by verified gross energy savings of high-impact measure population. No CHP participants completed a survey in PY10 and CHP is not included in the weighted estimate.

All projects in the Custom Program are unique and considered high impact measures, including

combined heat and power (CHP) projects. Commercial lighting contributes more than 5% to the sector

and portfolio and is considered a high impact measure. New measures offered in the Energy Efficient

Home Program are considered high impact measures; however, no participants who purchased these

measures completed a survey in PY10. Therefore, findings were determined using PY10 self-report

surveys for commercial lighting and custom. Overall, the PY10 high-impact measure NTG research

represents 40% of the total portfolio verified gross energy savings.

2.7 Summary of Energy Impacts by Program

Act 129 compliance targets are based on annualized savings (MWh/yr). Each program year, the annual

savings achieved by EE&C program activity are recorded as incremental annual, or “first-year” savings,

and added to an EDC’s progress toward compliance. Incremental annual savings estimates are presented

in the next section, 2.7.1 Incremental Annual Energy Savings by Program. Lifetime energy savings

incorporate the effective useful life (EUL) of installed measures and estimate the total energy savings

associated with EE&C program activity. Lifetime savings are used in the TRC test, by program

participants when assessing the economics of upgrades and by the statewide evaluator (SWE) when

calculating the emissions benefits of Act 129 programs.

Section 2.7.2 Lifetime Energy Savings by Program presents the lifetime energy savings by program.

2.7.1 Incremental Annual Energy Savings by Program

Figure 2-5 presents a summary of the program year-to-date (PYTD) energy savings by program for PY10.

The energy impacts in this report are presented at the meter and do not reflect adjustments for

transmission and distribution losses. The verified gross savings are adjusted by the energy realization

rate, and the verified net savings are adjusted by both the realization rate and the NTG ratio.


Figure 2-5. PY10 PYTD Energy Savings by Energy Efficiency Programs

Figure 2-6 presents a summary of the energy savings by program for Phase III of Act 129.

Figure 2-6. P3TD Energy Savings by Energy Efficiency Programs

A summary of energy impacts by program through PY10 is presented in Table 2-9. Demand response is

excluded from Figure 2-5 and Figure 2-6 and from this table because it does not produce energy savings.


Table 2-9. Incremental Annual Energy Savings by Program (MWh/Year)

Program PYRTD PYVTD PY

Unverified PYVTD

Net RTD VTD Unverified VTD Net

Appliance Recycling 14,295 11,362 - 7,499 39,784 33,938 - 22,399

Efficient Lighting 109,993 105,364 - 87,452 388,667 379,329 - 315,077

Energy Efficiency Kits and Education

12,083 9,304 - 9,304 34,708 30,352 - 30,352

Energy Efficient Home

20,434 17,661 - 11,593 52,760 46,407 - 32,477

Home Energy Education

42,079 42,829 - 42,829 116,422 113,387 - 113,387


190,754 184,990 - 136,442 505,765 490,939 - 363,488


6,075 6,011 - 6,011 16,790 16,573 - 16,573

WRAP 21,151 19,097 - 19,097 42,171 36,172 - 36,172

Total(1) 416,864 396,617 - 320,226 1,197,068 1,147,098 - 929,925

Adjustment for Home Energy Education Double-Counted Savings

(6,218) (16,534) -

Adjusted Portfolio Savings (1) (2) 390,399 - 1,197,068 1,130,564 - (1) Total may not match sum of rows due to rounding. (2) The adjusted verified savings in this table account for energy-savings uplift (double-counting) in the Home Energy Education Program.

The previously reported VTD savings from prior years, for the following programs, have changed since

the PY9 final annual report was submitted:

• Efficient Lighting. SWE audit activities recommended an adjustment of -282 MWh/year to the

PY9 gross/net verified savings due to SWE’s finding that not all candelabra-style lamps were

exempt from the Energy Independence and Security Act of 2007 (EISA 2007); thus they were

subject to different baseline assumptions than the evaluation ICSP used in estimating PY9 gross

verified savings.

• Home Energy Education. An adjustment of -96 MWh/year was made in the PY9 gross verified

savings due to calculation errors in the impact analysis. In addition, the adjustment for PY9

double-counted savings was changed from -5,938 to -5,921 MWh/year.

• Non-Residential Energy Efficiency. In the PY9 final annual report, 5 MWh/year of savings from

the Midstream Lighting component of this program were reported but not verified. Those

savings have since been verified with an energy realization rate of 0% and a net-to-gross ratio

(NTGR) of 0.85, which yields an additional 0 MWh/year of gross verified energy savings and an

additional 0 MWh/year of net verified energy savings.

Table 2-10 shows the verified savings for each program, by year reported and verified.


Table 2-10. Savings by Reported and Verified Year

Program Energy Savings (MWh/yr)

Reporting Year PY8 PY9 PY10

Verified Year PY8 PY9 PY9 PY10 PY10 Unverified

in PY10

Appliance Recycling 11,844 - 10,731 - 11,362 -

Custom 46,368 24,372 30,467 - 64,487 -

Efficient Equipment 70,917 - 115,994 - 96,197 -

Efficient Lighting 145,929 - 128,036 - 105,364 -

Energy Efficiency Kits and Education 9,219 - 11,829 - 9,304 -

Energy Efficient Home 9,943 - 18,802 - 17,661 -

Home Energy Education 34,326 - 36,232 - 42,829 -

Midstream Lighting - 1,917 15,915 - 24,306 -

Student Energy Efficient Education 4,539 - 6,024 - 6,011 -

WRAP 2,652 11 14,412 - 19,097 -

Total 335,739 26,299 388,442 - 396,617 -

2.7.2 Lifetime Energy Savings by Program

Table 2-11 presents the PYTD and P3TD lifetime energy savings by program. Lifetime savings are

adjusted to account for reduced lighting savings following the 2020 EISA backstop. Specifically, after the

2020 EISA implementation, year-one savings are reduced to the difference in energy usage between the

efficient bulb and the new baseline. No savings are included beyond 15 years, for any rebated item, per

the Pennsylvania TRC Order.12

Table 2-11. Lifetime Energy Savings

Program

PY10 Phase III

PYVTD Gross Lifetime (MWh/yr)

PYVTD Net Lifetime (MWh/yr)

VTD Gross Lifetime (MWh/yr)

VTD Net Lifetime (MWh/yr)

Appliance Recycling 86,448 57,055 245,775 161,835

Efficient Lighting 855,477 710,048 2,945,816 2,445,029


51,442 51,442 148,134 148,134

Energy Efficient Home 237,731 151,271 545,012 364,324

Home Energy Education 42,829 42,829 104,961 104,961


37,132 37,132 101,450 101,450


2,677,263 1,973,239 6,675,446 5,036,470

WRAP 123,155 123,155 192,581 192,581

Total(1) 4,111,477 3,146,171 10,959,177 8,554,785

12 The 2016 TRC Test Order for Phase III of Act 129 was adopted by PA PUC order at Docket No. M-2015-2468992 on June 11, 2015.


Program

PY10 Phase III

PYVTD Gross Lifetime (MWh/yr)

PYVTD Net Lifetime (MWh/yr)

VTD Gross Lifetime (MWh/yr)

VTD Net Lifetime (MWh/yr)


(6,218) (6,218) (15,264) (15,264)

Portfolio Total (1) (2) 4,105,258 3,139,953 10,943,913 8,539,521 (1) Total may not match sum of rows due to rounding. (2) The adjusted verified savings in this table account for energy-savings uplift (double-counting) in the Home Energy Education Program.

2.8 Summary of Demand Impacts by Program

PPL Electric Utilities’ Phase III EE&C programs achieve peak demand reductions in two primary ways. The

first is through coincident reductions from energy efficiency measures, and the second is through

dedicated demand response offerings that exclusively target temporary demand reductions on peak

days. Energy efficiency reductions coincident with system peak hours are reported and used in the

calculation of benefits in the TRC test, but they do not contribute to Phase III peak demand reduction

compliance goals. Phase III peak demand reduction targets are exclusive to demand response programs.

The two types of peak demand reduction savings are also treated differently for reporting purposes.

Peak demand reductions from energy efficiency are generally additive across program years, meaning

that the P3TD savings reflect the sum of the first-year savings in each program year. Demand reduction

stemming from energy efficiency programs does not contribute to the Act 129 demand response

requirements.

Demand response goals are based on average portfolio impacts across all events called in dedicated

demand response programs, so cumulative demand response performance is expressed as the average

performance of each of the demand response events called in Phase III to date.

Because of these differences, demand impacts from energy efficiency and demand response are

reported separately in the following subsections.

2.8.1 Energy Efficiency

Act 129 defines peak demand reductions from energy efficiency as the average expected reduction in

electric demand from 2:00 p.m. to 6:00 p.m. EDT on non-holiday weekdays from June through August.

Unlike Phase I and Phase II Act 129 reporting, the peak demand impacts from energy efficiency in this

report are presented at the meter and do not reflect adjustments for transmission and distribution

losses. Figure 2-7 presents a summary of the PYTD demand savings by energy efficiency program for

PY10.


Figure 2-7. PYTD Demand Savings by Energy Efficiency Program

Figure 2-8 presents a summary of the P3TD demand savings by energy efficiency program for Phase III of

Act 129.

Figure 2-8. P3TD Demand Savings by Energy Efficiency Program

Reported demand reduction for the Home Energy Education Program in PY8 were based on the demand reduction

reported in PY7, which were unreasonably high, skewing the demand realization rate for this program and for the portfolio

overall. PY9 reported demand reduction for this program use PY8 evaluated demand reduction.

A summary of the peak demand impacts by energy efficiency program through the current reporting

period is presented in Table 2-12.


Table 2-12. Peak Demand Savings by Energy Efficiency Program (MW/Year)

Program PYRTD PYVTD PY

Unverified PYVTD

Net RTD VTD Unverified VTD Net

Appliance Recycling 2.17 1.84 - 1.21 5.71 5.05 - 3.33

Efficient Lighting 14.92 13.55 - 11.24 55.08 50.74 0.00 42.15


0.82 0.95 - 0.95 2.47 2.93 - 2.93

Energy Efficient Home 3.59 3.14 - 1.98 9.22 8.47 - 5.59

Home Energy Education 7.29 7.23 - 7.23 68.22 20.09 - 20.09


26.07 27.70 - 20.59 67.41 69.10 0.00 51.08


0.59 0.62 - 0.62 1.62 1.73 - 1.73

WRAP 2.03 1.90 - 1.90 4.13 3.82 - 3.82

Total(1) 57.49 56.93 - 45.72 213.86 161.93 0.00 130.72


(0.91) (1.68)

Adjusted Total(1) (2) 57.49 56.01 - 213.86 160.24 0.00 (1) Total may not match sum of rows due to rounding. (2) The adjustment in this table account for uplift (double-counting) in the Home Energy Education Program.

The previously reported VTD savings from prior years, for the following programs, have changed since

the PY9 final annual report was submitted:

• Efficient Lighting. SWE audit activities recommended an adjustment of -0.04 MW/year to the

PY9 gross/net verified savings due to SWE’s finding that not all candelabra-style lamps were

exempt from the Energy Independence and Security Act of 2007 (EISA 2007); thus they were

subject to different baseline assumptions than the evaluation ICSP used in estimating PY9 gross

verified savings.

• Home Energy Education. An adjustment of -5.04 MW/year was made in the PY9 gross verified

savings due to calculation errors in the impact analysis. In addition, the adjustment for PY9

double-counted savings was changed from -0.93 to 0.91 MW/year.

2.8.2 Demand Response

Act 129 defines peak demand savings from demand response as the average reduction in electric

demand during the hours when a demand response event is initiated. Phase III demand response events

are initiated according to the following guidelines:

• Curtailment events shall be limited to the months of June through September.

• Curtailment events shall be called for the first six days of each program year (starting in PY10) in

which the peak hour of PJM’s day-ahead forecast is greater than 96% of its summer peak

demand forecast for the months of June through September.

• Each curtailment event shall last four hours.

• Each curtailment event shall be called such that it will occur during the day’s forecasted peak

hour(s) above 96% of the PJM RTO summer peak demand forecast.


• Once six curtailment events have been called in a program year, the peak demand reduction

program shall be suspended for that program year.

The peak demand impacts from demand response in this report are presented at the system level and

reflect adjustments to account for transmission and distribution losses. PPL Electric Utilities uses the

following line loss percentages/multipliers by sector:

• Residential = [8.75% or 1.0875]

• Small C&I = [8.75% or 1.0875]

• Large C&I = [4.20% or 1.042]

Table 2-13 summarizes the PYVTD and VTD demand reductions for the Demand Response Program in

the EE&C plan and for the demand response portfolio as a whole. VTD demand reductions are the

average performance across all Phase III demand response events independent of how many events

occurred in a given program year. The relative precision columns indicate the margin of error (at the

90% confidence interval) around the PYVTD and VTD demand reductions.

Table 2-13. Verified Gross Demand Response Impacts by Program

Program PYVTD Gross

MW

Relative

Precision (90%)

VTD Gross

MW

Relative

Precision (90%)

Demand Response 111.5 2.7% 116.6 2.1%

Portfolio Total 111.5 2.7% 116.6 2.1%

2.9 Summary of Fuel Switching Impacts

Act 129 allows EDCs to achieve electric savings by converting electric equipment to non-electric

equipment. Table 2-14 summarizes key fuel switching metrics to date in Phase III.

Table 2-14. Phase III Fuel Switching Summary

Fuel Switching Measures Offered

Electric to Fossil Fuel Central Heating

Electric to Fossil Fuel Water Heating

Custom Commercial Combined Heat and Power (CHP)

Custom Other Commercial Projects

Fuel Switching Measures Implemented

Electric to Fossil Fuel Central Heating- 245 projects

Custom Commercial Combined Heat and Power (CHP) - 7 projects

Custom Other Commercial Projects - 5 projects

VTD Energy Savings Achieved via Fuel

Switching (MWh/yr) 63,859 MWh/yr

P3TD Increased Fossil Fuel Consumption Due

to Fuel Switching Measures (MMBTU/yr) 328,418 MMBTU/yr

P3TD Incentive Payments for Fuel Switching

Measures ($1000) $2,259


2.10 Summary of Cost-Effectiveness Results

A detailed breakdown of portfolio finances and cost-effectiveness is presented in Table 2-15. TRC

benefits were calculated using gross verified impacts. Net present value (NPV) PY10 costs and benefits

are expressed in 2018 dollars. Net present value costs and benefits for P3TD financials are expressed in

2016 dollars.

Table 2-15. Summary of Portfolio Finances – Gross Verified

Row #

Cost Category PY10 ($1,000) P3TD ($1,000)

1 EDC Incentives to Participants (1) $23,087 $59,649

2 EDC Incentives to Trade Allies - -

3 Participant Costs (net of incentives/rebates paid by utilities) $62,400 $179,122

4 Incremental Measure Costs (Sum of rows 1 through 3) (9) $85,487 $238,770

EDC CSP EDC CSP

5 Design & Development (2) $1 - $450 $433

6 Administration, Management, and Technical Assistance (3) $2,029 $824 $6,744 $2,395

7 Marketing (4) $2,439 $1,983 $4,027 $6,836

8 Program Delivery (5) - $25,258 - $62,245

9 EDC Evaluation Costs $2,645 $7,865

10 SWE Audit Costs $400 $1,417

11* Program Overhead Costs (Sum of rows 5 through 10) (9) $35,578 $92,412

12 NPV of increases in costs of natural gas (or other fuels) for fuel switching programs

$10,150 $12,310

13 Total NPV TRC Costs (6) (9) (Net present value of sum of rows 4, 11, and 12)

$131,215 $343,493

14 Total NPV Lifetime Electric Energy Benefits $158,432 $399,528

15 Total NPV Lifetime Electric Capacity Benefits $32,211 $78,160

16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits

$20,365 $67,529

17 Total NPV Lifetime Non-Electric Benefits (Fossil Fuel, Water) $8,218 $6,396

18 Total NPV TRC Benefits (7) (Sum of rows 14 through 17) (9) $219,226 $551,613

19 TRC Benefit-Cost Ratio (8) (9) 1.67 1.61 (1) PPL Electric Utilities incorporates the cost of kits into the TRC as program delivery costs rather than incentives to participants. (2) Includes direct costs attributable to plan and advance the programs. Note: The design of the HERs program should be included here, while the actual development and mailing of HERs would be attributable to Program Delivery. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. Any common portfolio costs that are allocated across programs should be shown in this row. (4) Includes the marketing CSP and marketing costs by program CSPs. (5) Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. For behavioral programs, this includes the printing and postage of HERs. (6) Total TRC Costs includes Total EDC Costs and Participant Costs. (7) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (8) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (9) Total may not sum due to rounding. * Rows 1-11 are presented in nominal dollars (PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020); P3TD = 2016


TRC benefit-cost ratios are calculated by comparing the total NPV TRC benefits and the total NPV TRC

costs. It is important to note that TRC costs are materially different from the EDC spending and rate

recovery tables presented later in the report. TRC costs include estimates of the full cost incurred by

program participants to install efficient equipment, not just the portion covered by the EDC rebate.

Table 2-16 shows the TRC ratios by program and for the portfolio. The benefits were calculated using

gross verified impacts. PY10 benefits and costs are expressed in PY10 dollars as the analysis is

completed, using program years that align nominal calendar years values to a program year. The

Demand Response Program costs shown in Table 2-16 through Table 2-20 include those costs incurred

for PY10 after the Semi-Annual Report filed January 15, 2018.

Table 2-16. PY10 Gross TRC Ratios by Program ($1,000)

Program TRC NPV

Benefits (1) TRC NPV Costs (1) TRC Ratio

TRC Net Benefits (Benefits–Costs)

Appliance Recycling $4,189 $2,078 2.02 $2,111

Efficient Lighting $50,185 $7,441 6.74 $42,744

Energy Efficiency Kits and Education $6,640 $1,244 5.34 $5,396

Energy Efficient Home $17,074 $22,007 0.78 ($4,933)

Home Energy Education $2,499 $1,540 1.62 $959

Student Energy Efficient Education $7,214 $1,153 6.26 $6,061

WRAP $9,958 $8,702 1.14 $1,255

Residential (Including Low-Income) Subtotal (2) (3)

$97,759 $44,166 2.21 $53,593

Non-Residential Subtotal (2) $116,761 $77,025 1.52 $39,736

Demand Response $5,059 $2,283 2.22 $2,775

Common Portfolio Costs and Uplift ($353) $7,741 - ($8,094)

Portfolio Total (2) $219,226 $131,215 1.67 $88,010 (1) Costs and benefits are expressed as follows: PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020. (2) Total may not match sum of rows due to rounding. (3) Low-income is shown as a subsector of residential in this table.

Table 2-17 presents PY10 cost-effectiveness using net verified savings to calculate benefits. Net savings

for each program are calculated by multiplying the NTG ratios determined for the program sample to

the program verified energy savings. The adjustment for net savings impacts the total energy savings,

secondary energy savings, participant measure costs (reducing measure costs by NTGR), and operations

and maintenance (O&M) benefits.


Table 2-17. PY10 Net TRC Ratios by Program ($1,000)

Program TRC NPV



Appliance Recycling $2,765 $2,078 1.33 $687




Home Energy Education $2,499 $1,540 1.62 $959


WRAP $9,958 $8,702 1.14 $1,255


$81,795 $36,510 2.24 $45,285


Demand Response $5,059 $2,283 2.22 $2,775

Common Portfolio Costs and Uplift ($353) $7,741 - ($8,094)

Portfolio Total (2) $172,890 $103,864 1.66 $69,026 (1) Costs and benefits are expressed as follows: PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020 (2) Total may not match sum of rows due to rounding. (3) Low-income is shown as a subsector of residential in this table.

Table 2-18 summarizes cost-effectiveness by program for Phase III of Act 129. Benefits and costs are

expressed in PY8 dollars.

Table 2-18. P3TD Gross TRC Ratios by Program ($1,000)

Program TRC NPV


TRC Net Benefits (Benefits-Costs)





Home Energy Education $5,390 $3,682 1.46 $1,708


WRAP $15,512 $20,231 0.77 ($4,719)


$263,540 $117,843 2.24 $145,698


Demand Response $9,622 $4,004 2.40 $5,619

Common Portfolio Costs and Uplift ($722) $22,341 ($23,063)

Portfolio Total (2) $551,613 $343,493 1.61 $208,120 (1) Costs and benefits are expressed as follows: PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020 (2) Total may not match sum of rows due to rounding. (3) Low-income is shown as a subsector of residential in this table.


Table 2-19 presents P3TD cost-effectiveness results using net verified savings to calculate benefits.

Benefits and cost are expressed in PY8 dollars. Net savings for each program are calculated by

multiplying the NTG ratios determined for the program sample to the program verified energy savings.

The adjustment for net savings impacts the total energy savings, secondary energy savings, participant

measure costs, and O&M benefits. As noted in Table 2-7, NTG ratios determined in PY8 and PY9 were

used for some programs.

Table 2-19. P3TD Net TRC Ratios by Program ($1,000)

Program TRC NPV







Home Energy Education $5,390 $3,682 1.46 $1,708


WRAP $15,512 $20,231 0.77 ($4,719)


$222,689 $101,032 2.20 $121,658


Demand Response $9,622 $4,004 2.40 $5,619

Common Portfolio Costs and Uplift ($722) $22,341 ($23,063)

Portfolio Total (2) $439,047 $278,852 1.57 $160,195 (1) Costs and benefits are expressed as follows: PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020. (2) Total may not match sum of rows due to rounding. (3) Low-income is shown as a subsector of residential in this table.

2.11 Comparison of Performance to Approved EE&C Plan

Table 2-20 presents PY10 expenditures, by program, compared to the budget estimates set forth in the

EE&C plan for PY10.13 All of the dollars in this table are presented in PY10 dollars.

13 The EE&C Plan referenced in this section is PPL Electric Utilities revised Energy Efficiency and Conservation Plan Act 129 Phase III, EE&C plan (Docket No. M-2015-2515642), November 2018.


Table 2-20. Comparison of PY10 Expenditures to Phase III EE&C Plan ($1,000)

Program PY10 Budget

from EE&C Plan PY10 Actual

Expenditures Ratio

(Actual/Plan)

Appliance Recycling $2,345 $2,078 89%

Demand Response $3,227 $2,753 85%

Efficient Lighting $7,339 $5,283 72%

Energy Efficiency Kits and Education $1,500 $1,244 83%

Energy Efficient Home $6,650 $6,529 98%

Home Energy Education $1,930 $1,540 80%

Non-Residential Energy Efficiency $24,483 $21,642 88%

Student Energy Efficient Education $854 $1,153 135%

WRAP $10,955 $8,702 79%

Total Direct Program Costs (1) $59,282 $50,923 86%

Common Portfolio Costs (2) $8,620 $7,741 90%

Portfolio Total (1) $67,902 $58,664 86% (1) Total may not match sum of rows due to rounding. (2) Common costs include SWE costs.

Table 2-21 presents P3TD expenditures, by program, compared to the projected budget estimates set

forth in the EE&C plan through PY10. All dollars are presented in PY10 dollars.

Table 2-21. Comparison of P3TD Expenditures to Phase III EE&C Plan ($1,000)

Program Phase III Budget from EE&C Plan through PY10

Phase III Actual Expenditures through PY10

Ratio (Actual/Plan)

Appliance Recycling $6,565 $6,097 93%

Demand Response $6,028 $5,136 85%

Efficient Lighting $31,212 $26,561 85%

Energy Efficiency Kits and Education $4,170 $5,170 124%

Energy Efficient Home $16,878 $17,608 104%

Home Energy Education $5,535 $4,008 72%

Non-Residential Energy Efficiency $62,126 $50,121 81%

Student Energy Efficient Education $3,367 $3,160 94%

WRAP $25,826 $22,086 86%

Total Direct Program Costs (1) $161,708 $139,948 87%

Common Portfolio Costs (2) $25,860 $24,013 93%

Portfolio Total (1) $187,568 $163,961 87% (1) Total may not match sum of rows due to rounding. (2) Common costs include SWE costs.


Table 2-22 compares PY10 verified gross program savings compared to the energy savings projections

set forth in the EE&C plan.

Table 2-22. Comparison of PY10 Actual Program Savings to EE&C Plan Projections for PY10

Program PY10 EE&C Plan

(MWh/yr)

VTD Gross MWh/Yr Savings through PY10

(MWh/yr) Ratio (Actual/Plan)

Appliance Recycling 12,358 11,362 92%

Efficient Lighting 64,245 105,364 164%

Energy Efficiency Kits and Education 8,007 9,304 116%

Energy Efficient Home 14,408 17,661 123%

Home Energy Education 41,026 42,829 104%

Non-Residential Energy Efficiency 182,602 184,990 101%

Student Energy Efficient Education 2,816 6,011 213%

WRAP 14,716 19,097 130%

Total(1) 340,179 396,617 117%


(6,218)

Portfolio Total (1) (2) 340,179 390,399 115% (1) Total may not match sum of rows due to rounding. (2) The adjusted verified savings in this table account for energy-savings uplift (double-counting) in the Home Energy Education Program.

Table 2-23 compares Phase III verified gross program savings to the energy savings projections filed in

the EE&C plan.

Table 2-23. Comparison of Phase III Actual Program Savings to EE&C Plan Projections for Phase III

Program EE&C Plan through

PY10 (MWh/yr)

VTD Gross MWh/Yr Savings through PY10

(MWh/yr) Ratio (Actual/Plan)

Appliance Recycling 38,213 33,938 89%

Efficient Lighting 251,451 379,329 151%

Energy Efficiency Kits and Education 22,777 30,352 133%

Energy Efficient Home 31,706 46,407 146%

Home Energy Education 122,250 113,387 93%

Non-Residential Energy Efficiency 452,792 490,939 108%

Student Energy Efficient Education 13,175 16,573 126%

WRAP 37,180 36,172 97%

Total(1) 969,545 1,147,098 118%


(16,534)

Portfolio Total(1) (2) 969,545 1,130,564 117% (1) Total may not match sum of rows due to rounding. (2) The adjusted verified savings in this table account for energy-savings uplift (double-counting) in the Home Energy Education Program.


The reasons program savings varied from projections estimated in the EE&C Plan are summarized

below. Additional details can be found in the individual program sections.

• Appliance Recycling (residential sector). The Appliance Recycling Program achieved 92% of

projected energy savings. The program did not meet the projected savings for PY10 due to the

difference in ex ante and ex post per unit savings.

• Demand Response. In PY10, PPL Electric Utilities’ Demand Response Program achieved 121% of

the compliance target of 92 MW.

• Efficient Lighting (residential sector). The Efficient Lighting Program achieved 164% of its

projected energy savings for PY10 and a realization rate of 96%. Baseline wattage adjustments

increased program energy savings by roughly two percent. However, cross-sector sales

adjustments decreased program energy savings by about seven percent. Overall, these

adjustments produced a net 4% decrease in verified savings compared to reported savings.

• Energy Efficiency Kits and Education (residential low-income sector). The program savings

achieved 116% of the estimated projections for PY10, because the ICSP distributed 6,000 more

kits than the 8,000 kits estimated in the EE&C Plan for PY10. The ICSP switched to a ship-a-kit

system in PY10 so that agency clients could receive their kits directly in the mail. This process

change led to an estimated 58% increase in kits distributed through agencies. PPL Electric

Utilities and the ICSP made the decision to send more kits to increase the savings prior to

phasing out kits in PY12.

• Energy Efficient Home (residential sector). The program exceeded its projected energy savings

for the year, achieving 123% of the estimated projections. This is likely due to the program

experiencing higher participation than anticipated.

• Home Energy Education (residential sector). Cadmus verified 104% of the estimated projections

for PY10. The program exceeded its energy savings plans for the year because of increased

savings from the Phase III Expansion Wave and because the ICSP resumed treating low-

propensity customers from whom they suspended treatment in PY8 and most of PY9.

• Non-Residential. The Non-Residential Energy Efficiency Program exceeded its projected energy

savings for the year, achieving 101% of the estimated projections for PY10. The following factors

affected the program’s progress toward the estimated savings projected for PY10:

▪ The Lighting and Equipment components achieved verified savings of 52% of total program

projected savings for PY10, at a realization rate of 97% for lighting and 81% for equipment.

▪ The Custom component achieved verified energy savings that contributed 35% of projected

savings for PY10, at a realization rate of 95%.

▪ The Midstream component contributed verified savings of 13% to the program, at a

realization rate of 103%.

▪ The GNE sector rebates were put on a waitlist in January 2018 because participation rates

were higher than expected in the first two years of Phase III.


• Student Energy Efficient Education (residential sector). The Student Energy Efficient Education

Program exceeded its projected energy savings for the year due to substantially greater

participation in PY10 (23,665 participants) than planned (13,047 participants). PPL Electric

Utilities’ EE&C Plan proposed adding a low-income Student Energy Efficient Education offering

to its portfolio in PY10. The ICSP targeted schools with low-income students, increasing program

participation in PY10. A percentage of the program’s savings (52%) were attributed to the

low-income sector, determined using Pennsylvania Department of Education data specifying the

percentage of students receiving reduced-fee and free lunches.

• WRAP (residential low-income sector). The program’s verified savings met 130% of estimated

savings projected for PY10. The program-verified savings were better than projections due to

increased participation and improvements in program delivery.

Program Changes for PY11

PPL Electric Utilities has made the following program changes:

• Student Energy Efficient Education. The PA PUC approval of changes to PPL Electric Utilities’

EE&C Plan (received in PY11) included the addition of a program component targeting low-

income schools. The change reallocated a portion of the program’s energy savings, budget, and

participants from the residential program to the low-income sector. In PY11, this program is

focusing on schools in low-income areas of PPL Electric Utilities’ service territory with a

minimum of 45% reduced and free lunches, as documented by the Pennsylvania Department of

Education. Savings for this low-income component will be reported under WRAP.

• Home Energy Education. The program was modified in PY11 to add a low-income component.

PPL Electric Utilities is offering home energy reports (HERs) to low-income customers with

specific messaging and tips in the HERs to encourage low-income customers to enroll in PPL

Electric Utilities’ low-income programs (WRAP and OnTrack). Savings for this low-income

component will be reported under WRAP.

• Custom. In PY11, the Custom Program expanded to offer direct install custom projects.

Customer eligibility remains consistent to GS1 and GS3 rate classes. GNE customers are not

eligible because projects for this sector as a whole are on a wait list. Preapproval is required

prior to construction, and a kickoff call with the engineering team is held to fully discuss the

scope of work and calculation of savings. The incentive is $0.14/kWh saved for direct install

custom projects installed through the Custom Program, with projects capped at 80% of the

project costs.

2.12 Summary of Process Evaluation Results

This section summarizes program satisfaction results gathered from the participant surveys. Table 2-24

lists the programs for which Cadmus conducted participant surveys in PY10 and the number of

respondents who answered the program satisfaction question. Details on each program’s survey

methodology are provided in the program chapters and their respective appendices.


Table 2-24. PY10 Participant Surveys and Program Satisfaction Response Counts

Sector and Program Survey Mode Targeted Number of Completed Surveys

Number of Satisfaction Responses (1)

Residential Sector 18,625

Appliance Recycling Online All records (3,078) 379

Energy Efficient Home Equipment Online All records (2,357)

495 Energy Efficient Home Weatherization Online All records (517)

Energy Efficient Home In-home Audit Online All records (101)

Energy Efficient Home Online Assessment Online All records (5,796)

Home Energy Education Treatment Telephone 250

512 Online 250

Student Energy Efficient Education Home Energy Worksheets All returned surveys

(23,665) 17,239

Non-Residential Sector 105

Continuous Energy Improvement Telephone 4 4

Custom Telephone and Online All records (38) 21

Efficient Equipment Telephone and Online 68 67

Midstream Lighting Participating Distributors 15 13

Low-Income Sector 1,093

Energy Efficiency Kits and Education Returned Kit Surveys All records (13,564) 938

WRAP Telephone 155 155

Portfolio 19,823 (1) The total number of responses in this table includes completed and partially completed surveys. Not all survey respondents answered the program satisfaction question because respondents can refuse to answer. Because of these reasons, the total number of responses in this table and Table 3-3 may not match each other.

2.12.1 Portfolio-Level Program Satisfaction

Throughout this report, Cadmus refers to the PY8 and PY9 reports when comparing results.14,15

Cadmus asked respondents how satisfied they were with the program overall, using a 5-point word scale

from very satisfied to not at all satisfied, with a neutral midpoint. Cadmus combined the percentages of

respondents who rated their satisfaction with the program as very satisfied or somewhat satisfied and

computed a straight average of all programs to determine the portfolio-level and sector-level program

satisfaction results.

Figure 2-9 shows that at a portfolio-level average, PY10 achieved high program satisfaction.

14 PPL Electric Utilities. Annual Report Program Year 8: June 1, 2016–May 31, 2017. Presented to Pennsylvania Public Utility commission. Prepared by Cadmus. November 15, 2017. Available online: http://www.puc.pa.gov/pcdocs/1544671.pdf

15 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to Pennsylvania Public Utility Commission. Prepared by Cadmus. November 15, 2018. Available online: http://www.puc.pa.gov/pcdocs/1595564.pdf

http://www.puc.pa.gov/pcdocs/1544671.pdf



Figure 2-9. Portfolio-Level Program Satisfaction Very and Somewhat Satisfied Combined

The program satisfaction results include all responses to the satisfaction question, averaged to compute the portfolio level

satisfaction. The percentage in the white box indicates that the difference between PY10 and PY9 is statistically significant,

p≤0.10. Source: Participant survey question, “How would you rate your overall satisfaction with the program?” The PY9 Annual

Report included the results for Demand Response but the results for Demand Response are not included in any of the program

years in this figure. The number of surveys included in the PY9 satisfaction calculation was reported incorrectly in the PY9

Annual Report but has been corrected in this figure

2.12.2 Program Satisfaction by Sector

For Phase III, PPL Electric Utilities established a sector-level satisfaction goal to achieve 80% or greater of

very satisfied and somewhat satisfied customers.16 As shown in Figure 2-10, respondents across all three

sectors showed high program satisfaction and exceeded the customer satisfaction goal of 80% or

greater. The low-income sector achieved the highest percentage of satisfied respondents at 95%

(n=1,093), compared to 91% for the nonresidential sector (n=105) and 83% for the residential sector

(n=18,625). The nonresidential sector in PY10 observed a significant decrease in program satisfaction

from PY9.17

16 The customer satisfaction goal is stipulated in PPL Electric Utilities’ revised EE&C Plan (Docket No. M-2015-2515642) filed with the PA PUC, November 2018.

17 Difference is statistically significant, p≤0.10.


Figure 2-10. PY10 Program Satisfaction by Sector Very and Somewhat Satisfied Combined

The percentage in the black box indicates that the difference between PY9 and PY10 is statistically

significant, at p≤0.10 or better. The program satisfaction results include all responses to the satisfaction

question. Low-income sector satisfaction was reported incorrectly in the PY9 Annual Report.

Source: Participant survey question, “How would you rate your overall satisfaction with the program?”

2.12.3 Program Satisfaction by Individual Program

Figure 2-11 shows the satisfaction results for each program. The Appliance Recycling and Energy

Efficiency Kits and Education programs achieved the highest satisfaction (97%, n=379 and n=938,

respectively). The Home Energy Education Program had the lowest satisfaction (66%, nw=512). Note

that this type of program typically receives some of the lowest satisfaction scores because of the

opt-out participation design and because it does not offer the incentives that traditional rebate

programs offer.

Further details on each program’s satisfaction results are provided in the individual program chapters.


Figure 2-11. PY10 Program Satisfaction by Individual Program Very and Somewhat Satisfied Combined

The program satisfaction results include all responses to the satisfaction question.

Home Energy Education uses the notation “nw” to indicate that survey results were weighted.

Source: Participant survey question, “How would you rate your overall satisfaction with the program?”

2.13 Findings and Recommendations

The impact and process evaluation activities completed by Cadmus led to recommendations for

program improvement. Cadmus does not have any overarching recommendations that affect more than

one program. Specific recommendations for each program are in the program chapters.

Chapter 3 Evaluation Results by Program PPL Electric Utilities | 36

3 Evaluation Results by Program

This chapter documents the gross impact, net impact, and process evaluation activities conducted in

PY10, along with the outcomes of those activities. The individual program chapters are organized by the

largest contributor to PY10 portfolio savings to the smallest. Program information in portfolio-level

tables are organized in alphabetical order.

Table 3-1 lists the activities for each program in PPL Electric Utilities’ portfolio.

Table 3-1. PY10 Evaluation Activity Matrix

Program Sector Gross Impact Net Impact Process

Appliance Recycling Residential ✓ ✓ ✓

Demand Response Demand Response ✓ ✓

Energy Efficient Home Residential ✓ ✓ ✓

Energy Efficiency Kits and Education Low-Income ✓ ✓

Efficient Lighting Residential ✓ ✓(1) ✓

Home Energy Education Residential ✓ ✓

Non-Residential Energy Efficiency Nonresidential ✓ ✓ ✓

Student Energy Efficient Education Residential ✓ ✓

WRAP Low-income ✓ ✓

(1) Cadmus did not complete a demand elasticity study but did update the market progress indicator analysis.

3.1 Impact Evaluation

Impact evaluation activities varied by program in PY10. More detailed explanations of each program’s

impact evaluation methodology and analyses are contained in the program chapters and their

respective appendices. Table 3-2 lists the impact evaluation activities conducted for each program in

PY10 along with the number of site visits conducted for each program. The individual program chapters

discuss the impact evaluation activities, methodology, and findings.


Table 3-2. PY10 Impact Evaluation Activities by Program

Program

Impact Evaluation Activity

Database Review

Records Review

Desk Audits

Site Visits (1)

Metering Engineering

Analysis Billing

Analysis

Appliance Recycling ✓ ✓ ✓

Demand Response ✓ ✓ ✓

Energy Efficient Home ✓ ✓ ✓


✓ ✓ ✓

Efficient Lighting ✓ ✓ ✓

Home Energy Education ✓ ✓ ✓

Non-Residential - Continuous Energy Improvement

✓ ✓ ✓

Non-Residential - Custom

✓ ✓ ✓ (2) ✓ ✓ ✓

Non-Residential - Efficient Equipment

✓ ✓ ✓ ✓ (3) ✓ ✓

Non-Residential - Midstream Lighting

✓ ✓ ✓ ✓ (4) ✓ ✓


✓ ✓ ✓

WRAP ✓ ✓ ✓ (1) Site visits completed by Cadmus. (2) Includes 37 visits. (3) Includes 17 equipment visits and 25 lighting visits. (4) Includes 14 site visits for 87 projects.

3.2 Process Evaluation

This section summarizes the process evaluation of PPL Electric Utilities’ PY10 portfolio.

The individual program process evaluations identify opportunities and offer recommendations to

improve the overall effectiveness of the design, implementation, enrollment process, quality assurance,

and other elements for all of PPL Electric Utilities’ energy efficiency programs.

Each program assessment is discussed in more detail in individual chapters of this report. The chapters

discuss the findings from the program-specific evaluation activities and note any modifications to these

activities from Cadmus’ evaluation plans.

Table 3-3 lists the process evaluation activities conducted for each program in PY10, along with the total

number of survey and interview respondents reached for each program. A more detailed explanation of

each programs’ survey methodology is in the program chapters and their respective appendices.


Table 3-3. PY10 Process Evaluation Activities by Program

Program Completed Participant Survey (1)

Completed General

Population Survey

Logic Model Review

Participant Satisfaction

Analysis

Stakeholder Interview

Trade Ally Interview

Market Actor

Interview

Appliance Recycling 368 ✓ ✓

Demand Response 12 ✓ ✓

Energy Efficient Home 488 (2) ✓ ✓


948 (3) ✓ ✓ ✓ 5 (4)

Efficient Lighting 698 (5) ✓

Home Energy Education 414 ✓ ✓

Non-Residential - Continuous Energy Improvement

4 ✓ ✓

Non-Residential - Custom 21 ✓ ✓

Non-Residential - Efficient Equipment

68 (6) ✓ ✓

Non-Residential - Midstream Lighting

13 (7) ✓ ✓ 14 (8)


17,239 (9) ✓ ✓

WRAP 155 ✓ ✓ ✓ 3 (10)

Total 20,442 698 N/A N/A N/A 19 3 (1) Includes all survey modes: online, telephone, and paper. For additional detail, see program chapter and appendix. This does not include partially completed surveys and may not match the totals used to calculate program satisfaction. (2) Includes 210 equipment, 6 in-home audit, 234 online assessment, and 38 weatherization surveys. (3) Includes 753 direct mail paper surveys and 195 agency paper surveys administered by the ICSP. (4) Includes 5 community based organizations. (5) Includes 300 residential surveys and 398 small business surveys. (6) Includes 7 equipment, 24 direct discount lighting, and 37 prescriptive lighting surveys. (7) These are participating distributors. (8) Includes 7 end-user purchasers and 7 contractor purchasers. (9) Includes 17,239 paper and online home energy worksheets administered by the ICSP. (10) Includes 3 master-metered multifamily property managers.

Chapter 4 Portfolio Finances and Cost Recovery PPL Electric Utilities | 39

4 Portfolio Finances and Cost Recovery

This section provides an overview of the expenditures associated with PPL Electric Utilities’ portfolio and

the recovery of those costs from ratepayers.

4.1 Program Finances

Program-specific and portfolio total finances for PY10 are shown in Table 4-1 and for and for Phase III in

Table 4-2. The column headings in these tables are adapted from the Direct Program Cost categories in

the PA PUC’s template for the EE&C plan for Phase III.18 The column titled EDC Materials, Labor, and

Administration includes costs associated with an EDC’s own employees. The column headed ICSP

Materials, Labor, and Administration includes both the program implementation contractor and the

costs of any other outside vendors and EDCs employed to support program delivery. The dollar amounts

are based on EDC tracking of expenditures with no adjustments to account for inflation.19

Table 4-1. PY10 Program and Portfolio Total Finances

Program Incentives to

Participants and Trade Allies

EDC Materials, Labor, and

Administration

ICSP Materials, Labor, and

Administration EM&V Total [1]

Appliance Recycling Program $404 $46 $1,628 - $2,078

Demand Response Program $1,879 $41 $833 - $2,753

Efficient Lighting Program $3,330 $41 $1,911 - $5,283

Energy Efficiency Kits & Education Program [2] - $48 $1,196 - $1,244

Energy Efficient Home Program $3,189 $41 $3,299 - $6,529

Home Energy Education Program $40 $1,500 - $1,540

Non-Residential Energy Efficiency $14,286 $171 $7,184 - $21,641

Student Energy Efficiency Education Program - $26 $1,128 - $1,153

WRAP [2] - $197 $8,505 - $8,702

Common Portfolio Costs (3) - $3,817 $879 $2,645 $7,341

Portfolio Total [3] (4) $23,087 $4,468 $28,063 $2,645 $58,264

SWE Costs (5) $400

Total [4] $23,087 $4,468 $28,063 $2,645 $58,664 [1] Total may not sum due to rounding. [2] Costs associated with low-income program measures provided to customers at no cost are categorized as administrative costs. [3] Common Portfolio Costs are costs applicable to more than one customer class, to more than one program, or those that provide portfolio-wide benefits. These include PPL Electric Utilities labor and materials, costs related to PPL Electric Utilities’ tracking system, EE&C plan development, etc. [4] Portfolio Total and Total may not equal total of column due to rounding. [5] Statewide Evaluation costs are outside of the 2% spending cap.

18 Pennsylvania Public Utility Commission. July 21, 2015. Implementation of Act 129 of 2008—Phase III Energy Efficiency and Conservation Plan Template Docket No. M-2014-2424864. Section 10.

19 The cost-recovery of program expenses through riders generally happens promptly so that costs are being recovered from ratepayers in the same dollars that they are incurred.


Program-specific and portfolio total finances since the inception of Phase III are shown in Table 4-2.

Table 4-2. P3TD Program and Portfolio Total Finances

Program Incentives to

Participants and Trade Allies

EDC Materials, Labor, and

Administration

ICSP Materials, Labor, and

Administration EM&V Total [1]

Appliance Recycling Program $1,108 $124 $4,866 - $6,097

Demand Response Program $2,858 $243 $2,035 - $5,136

Efficient Lighting Program $21,396 $179 $4,986 - $26,561

Energy Efficiency Kits & Education Program [2] - $150 $5,020 - $5,170

Energy Efficient Home Program $7,442 $160 $10,006 - $17,608

Home Energy Education Program - $103 $3,905 - $4,008

Non-Residential Energy Efficiency $31,319 $590 $18,213 - $50,121

Student Energy Efficiency Education Program - $152 $3,009 - $3,160

WRAP [2] - $662 $21,423 - $22,086

Common Portfolio Costs [3] - $9,767 $4,255 $8,492 $22,513

Portfolio Total [3] [4] $64,123 $12,129 $77,718 $8,492 $162,461

SWE Costs [5] $1,500

Total [4] $64,123 $12,129 $77,718 $8,492 $163,961 [1] Total may not sum due to rounding. [2] Costs associated with low-income program measures provided to customers at no cost are categorized as administrative costs. [3] Common Portfolio Costs are costs applicable to more than one customer class, to more than one program, or those that provide portfolio-wide benefits. These include PPL Electric Utilities labor and materials, costs related to PPL Electric Utilities’ tracking system, EE&C plan development, etc. [4] Portfolio Total and Total may not equal total of column due to rounding. [5] Statewide Evaluation costs are outside of the 2% spending cap.

4.2 Cost Recovery

Act 129 allows Pennsylvania EDCs to recover EE&C plan costs through a cost-recovery mechanism. PPL

Electric Utilities’ cost-recovery charges are organized separately by customer sectors to ensure that the

electric rate classes that finance the programs are the rate classes that receive the direct energy and

conservation benefits. Cost-recovery is governed by tariffed rate class, so it is necessarily tied to the way

customers are metered and charged for electric service.

Table 4-3 shows PPL Electric Utilities’ EE&C Plan Expenditures for PY10 and Phase III.


Table 4-3. EE&C Plan Expenditures by Cost-Recovery Category(1) ($1,000)

Cost Recovery

Customer Sector Rate Schedules Included PYTD Spending P3TD Spending

Residential & Low-Income Residential (primarily RS) $25,647 $84,295

Small C&I Small C&I (primarily GS1 & GS3) $12,877 $26,232

Large C&I Large C&I (primarily LP4 & LP5) $9,484 $23,492

GNE Residential, Small C&I, and Large C&I $4,840 $12,835

Common [2] - $5,815 $17,107

Portfolio Total [3] - $58,664 $163,961

[1] Includes SWE costs. [2] Includes costs not collected at the sector level. These costs are allocated to the sectors at the end of the phase. [3] Totals may not sum due to rounding.

Chapter 5 Non-Residential Energy Efficiency Program PPL Electric Utilities | 42

5 Non-Residential Energy Efficiency Program

PPL Electric Utilities' Non-Residential Energy Efficiency Program offers financial incentives to customers

in a nonresidential rate class and for any building or business type. The program comprises four distinct

components—Efficient Equipment, Midstream Lighting, Custom, and Continuous Energy Improvement

(CEI). For this evaluation, Cadmus treated each of these components as an individual program offering

and designed a distinct set of data collection activities, research and evaluation methodologies.

Descriptions of the Non-Residential Energy Efficiency Program components and the evaluation

methodology, findings, conclusions and recommendations for each are provided in separate chapters.

• Chapter 6 Non-Residential Efficient Equipment component offers prescriptive rebates and

direct discounts to small businesses for lighting and equipment products.

• Chapter 7 Non-Residential Midstream Lighting component offers incentives to distributors of

efficient lighting products for eligible products sold to PPL Electric Utilities’ customers.

• Chapter 8 Non-Residential Custom component provides financial incentives to customers who

install products or offer services that are not offered in PPL Electric Utilities’ other programs.

• Chapter 9 Non-Residential Continuous Energy Improvement (CEI) initiative provides technical

support for schools to develop and implement a strategic energy management plan (SEMP).

The objectives of the Non-Residential Energy Efficiency Program are these:20

• Provide energy-saving opportunities to qualified customers

• Increase the market penetration of high-efficiency technologies and building systems for

customers by offering incentives for high-efficiency and ENERGY STAR-rated appliances, lighting

equipment, and HVAC systems

• Encourage customers to take a comprehensive, whole-facility approach to energy efficiency by

installing high-efficiency custom measures or processes

• Encourage qualifying equipment repairs, optimization, and operational or process changes that

reduce electricity consumption

• Increase customer awareness of the features and benefits of energy-efficient equipment

• Support emerging technologies and nontypical efficiency solutions in cost-effective applications

• Encourage advanced energy efficiency strategies required for certification by national market

transformation programs such as Leadership in Energy and Environmental Design (LEED),

Architecture 2030, or ENERGY STAR Buildings

• Engage trade allies to stock, promote, and provide high-efficiency technology options to

customers

20 Program objectives are stipulated in PPL Electric Utilities Corporation. Energy Efficiency and Conservation Plan Act 129 Phase III. Docket No. M-2015-2515642 Compliance Filing before the Pennsylvania Public Utility Commission. November 2018.


• Promote other PPL Electric Utilities energy efficiency programs

• Collect energy and operating data from customers, as required to confirm customer and

measure eligibility, and to determine energy savings and cost-effectiveness

• Obtain participation necessary to achieve approximately 810,810 MWh/year gross verified

savings

• Achieve high customer and trade ally satisfaction with the program

5.1 Gross Impact Evaluation

Table 5-1 shows the Non-Residential Energy Efficiency Program’s verified gross savings.

Table 5-1. Non-Residential Energy Efficiency Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 143,573 162,377 184,990 490,939 (1)

(1) Phase III verified savings may not match sum of program years due to rounding.

The impact and process evaluation findings for each non-residential component are described in the

component’s individual chapter. Table 5-2 presents the participation counts, reported and verified

energy and demand savings, and incentive payments across all components of the Non-Residential

Energy Efficiency Program in PY10 by customer segment.

Table 5-2. PY10 Non-Residential Energy Efficiency Program Participation and Reported Impacts

Parameter GNE Large C&I

(Non-GNE) Residential

Small C&I (Non-GNE)

Total ⁽¹⁾

PYTD # Participants 2,247 936 253 5,322 8,758

PYRTD MWh/yr 45,360 65,133 450 79,811 190,754

PYRTD MW/yr 5.35 8.53 0.09 12.10 26.07

PYVTD MWh/yr 45,423 63,010 459 76,098 184,990

PYVTD MW/yr 7.62 8.15 0.09 11.85 27.70

PY10 Incentives ($1000) $2,543 $3,861 $44 $7,838 $14,286

⁽¹⁾ Total may not match sum of columns due to rounding.

Cadmus calculated gross verified savings using data from the PPL Electric Utilities tracking database and

from a combination of evaluation activities, including records review, desk audits, engineering analyses,

site visits, and billing analysis. Table 5-3 shows the gross energy and demand savings realization rates for

the components of the Non-Residential Energy Efficiency Program in PY10.


Table 5-3. PY10 Non-Residential Energy Efficiency Program Gross Energy Savings Realization Rates by Component

Component PYRTD

MWh/yr PYRTD MW/yr

Energy Savings Realization

Rate

Demand Savings

Realization Rate

PYVTD MWh/yr

PYVTD MW/yr

Efficient Equipment - Lighting

96,138 14.18 97% 98% 93,138 13.95

Efficient Equipment - Equipment

3,772 0.27 81% 87% 3,059 0.23

Midstream Lighting 23,542 4.46 103% 96% 24,306 4.27

Custom 67,159 7.16 95% 127% 63,938 9.09

Continuous Energy Improvement

144 0 382% - 549 0.17

Total (1) 190,754 26.07 97% 106% 184,990 27.70

(1) May not sum due to rounding.

Table 5-4 and Table 5-5 show the Non-Residential Energy Efficiency Program’s PY10 total reported

energy savings and demand reduction, respectively.

Table 5-4. PY10 Non-Residential Energy Efficiency Program Gross Impact Results for Energy

PYRTD MWh/yr

Energy Realization

Rate

Sample Cv or Error Ratio

Relative Precision at

85% C.L.

PYVTD MWh/yr (1)

Program Total 190,754 97% N/A 3.42% 184,990 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Table 5-5. PY10 Non-Residential Energy Efficiency Program Gross Impact Results for Demand

PYRTD MW/yr

Demand Realization

Rate



85% C.L.

PYVTD MW/yr (1)

Program Total 26.07 106% N/A 5.04% 27.70 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

5.2 Net Impact Evaluation

Table 5-6 shows the NTG ratios for the Non-Residential Energy Efficiency Program components in PY10.


Table 5-6. PY10 Non-Residential Energy Efficiency Program NTG Ratios by Component

Component NTG Ratio Program Verified

Gross MWh/yr

Percentage of Total Program Verified

Gross MWh/yr

Program Verified Net

MWh/yr

Efficient Equipment - Lighting 0.77 93,138 50% 71,716

Efficient Equipment - Equipment 0.64 3,059 2% 1,958

Midstream Lighting (1) 0.85 24,306 13% 20,660

Custom 0.65 63,938 35% 41,560

Continuous Energy Improvement 1.00 549 0.3% 549

Total (2) 0.74 184,990 136,442 (1) NTG ratio was not calculated in PY10. PY9 NTG ratio was applied in PY10. (2) May not sum due to rounding.

5.3 Verified Savings Estimates

Table 5-7 shows the reported energy savings (PYRTD) and verified gross and net energy savings

estimates for the Non-Residential Energy Efficiency Program in PY10.

Table 5-7. PYTD and P3TD Non-Residential Energy Efficiency Program Savings Summary

Savings Type Energy (MWh/yr) ⁽¹⁾ Total Demand (MW/yr) ⁽¹⁾

PYRTD 190,754 26.07

PYVTD Gross 184,990 27.70

PYVTD Net ⁽²⁾ 136,442 20.59

P3RTD 505,765 67.41

P3VTD Gross 490,939 69.10

P3VTD Net ⁽²⁾ 363,488 51.08

(1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy savings compliance target.

5.4 Cost-Effectiveness Reporting

A detailed breakdown of program finances and cost-effectiveness is presented in Table 5-8. The total

resource cost (TRC) benefits were calculated using gross verified impacts. Net present value (NPV) PYTD

costs and benefits are expressed in PY10 dollars (PY10 includes months in both 2018 and 2019). NPV

costs and benefits for P3TD financials are expressed in the PY8 dollars. The TRC costs and benefits in this

table do not include costs and benefits from unverified projects.

Cadmus quantified non-energy benefits in accordance with the SWE’s Guidance Memo.21 A summary of

the methodologies Cadmus used to calculate the non-energy benefits of natural gas savings is presented

in Appendix P Non-Energy Benefits.

21 Guidance on the Inclusion of fossil fuel and H2O benefits in the TRC Test, Statewide Evaluation Team, March 25, 2018.


Table 5-8. Summary of Program Finances – Gross Verified

Row # Cost Category PYTD ($1,000) P3TD ($1,000) (10)

1 EDC Incentives to Participants $14,286 $28,787




5 Design & Development (2) - - - -

6 Administration, Management, and Technical Assistance (3) $171 - $555 -

7 Marketing (4) - $322 - $1,477


9 EDC Evaluation Costs - -

10 SWE Audit Costs - -

11(6) Program Overhead Costs (Sum of rows 5 through 10) (1) $7,356 $17,393


$3,981 $6,035

13 Total NPV TRC Costs (Net present value of sum of rows 4, 11, and 12) (7)

$77,025 $199,305



16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits $4,231 $10,827

17 Total NPV Lifetime Non-Electric Benefits (Fossil Fuel, Water) ($6,615) ($11,413)

18 Total NPV TRC Benefits (Sum of rows 14 through 17) (8) $116,761 $279,172

19 TRC Benefit-Cost Ratio (9) 1.52 1.40 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio-level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs. (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.

Table 5-9 presents program financials and cost-effectiveness on a net savings basis.


Table 5-9. Summary of Program Finances – Net Verified








7 Marketing (4) - $322 - $1,477






$2,588 $4,119


$57,330 $151,475






19 TRC Benefit-Cost Ratio (9) 1.51 1.37 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio-level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs. (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.

EFFICIENT EQUIPMENT PROGRAM

PY10 PARTICIPATION

Prescriptive Lighting

Direct Discount Lighting

Equipment

253

707


72


PY10 saved

283,108 MWh/yr

96,197 MWh/yr

58%

52%

of Non-Residential program savings

of Non-Residential program savings


This program promotes the purchase and installation of high-efficiency equipment and lighting by offering customers financial incentives to offset purchase costs and by providing information on efficiency features and benefits.



72%

92%

96% satisfied with Direct Discount Lighting

satisfied with professionalism of the program representatives

satisfied with the information provided about the application process

satisfied with Equipment

satisfied with Prescriptive Lighting

satisfied with the time it took to process their application

satisfied with the ease of the online application process

satisfied with the ability to track their rebates

79%

87%

89%

73%

74%

Chapter 6 Non-Residential Efficient Equipment Program PPL Electric Utilities | 49

6 Non-Residential Efficient Equipment Program

The Efficient Equipment component of the Non-Residential Energy Efficiency Program (hereafter

referred to as the Efficient Equipment Program) promotes the purchase and installation of high-

efficiency equipment and lighting by offering customers financial incentives to offset the higher

purchase costs of such equipment and by providing information on their features and benefits. This

program targets small C&I, large C&I, GNE, and agricultural customers.

The program offers incentives for lighting and equipment (HVAC, refrigeration, motors, food service,

office, and agricultural) through two delivery channels—prescriptive and direct discount.

Prescriptive delivery channel. In the prescriptive delivery channel, the customer installs the equipment,

submits the rebate application, and receives the rebate. For all equipment offered through the Efficient

Equipment Program, PPL Electric Utilities provides incentives in the range of $0.02 to $0.17 per annual

kWh saved. Incentives may be capped at 50% to 100% of the total project costs (excluding internal

labor), with a maximum incentive of $500,000.

Direct discount delivery channel. The direct discount delivery channel was designed to make it easier

and more economical for small businesses and institutions to install energy-efficient lighting fixtures and

controls, commercial refrigeration equipment and controls, and compressed air system upgrades. This

channel does not have a maximum energy savings cap but is limited to small commercial and industrial

facilities with GS-1 or GS-3 rate codes. Through this channel, a contractor evaluates possible upgrades

and makes recommendations. The customer chooses which projects to install, and the contractor

completes and submits the required paperwork on the customer’s behalf to PPL Electric Utilities. The

customer pays the contractor for the discounted equipment up front, thereby lowering the overall

cost burden. PPL Electric Utilities awards the incentive to the contractor who has already passed the

cost savings to the customer.

In this report, projects are referred to as either lighting or equipment (non-lighting). The report is

organized first by lighting and then by equipment.

6.1 Participation and Reported Savings by Customer Segment – Lighting

6.1.1 Definition of a Lighting Participant

A prescriptive lighting participant is defined as a unique job initiated by a customer. In PY10, the

prescriptive lighting channel had 707 lighting jobs (11,422 individual database records) and 653 unique

customers.

A direct discount lighting participant is defined as a unique job completed for a unique customer. In

PY10, the direct discount lighting delivery channel had 253 jobs (1,753 individual database records) and

244 unique customers. In PY9, Cadmus evaluated the lighting jobs from the direct discount delivery

channel as a separate stratum from the prescriptive lighting jobs. In PY10, Cadmus grouped the direct

discount lighting jobs with the prescriptive lighting stratum because the PY9 evaluation did not find

meaningful differences in Cv or realization rates between the two delivery channels.


6.1.2 Program Participation and Reported Impacts for Lighting

Table 6-1 presents the participation counts, reported energy and demand savings, and incentive

payments for the lighting portion of the Efficient Equipment Program in PY10, by customer segment.

Table 6-1. PY10 Efficient Equipment Program (Lighting) Participation and Reported Impacts

Parameter GNE Large C&I Residential Small C&I Total (1)

PYTD # Participants 96 100 1 763 960

PYRTD MWh/yr 7,572 33,451 24 55,091 96,138

PYRTD MW/yr 1.41 4.55 0.00 8.22 14.18

PYVTD MWh/yr 7,478 32,507 22 53,130 93,138

PYVTD MW/yr 1.47 4.28 0.00 8.19 13.95

PY10 Incentives ($1000) (2)

(1) May not match due to rounding. (2) Incentives are tracked at the program level and reported in findings for the Non-Residential Energy Efficiency Program

6.2 Gross Impact Evaluation – Lighting

6.2.1 Gross Impact Evaluation Activities - Lighting

The evaluation sampling strategy is shown in Table 6-2. See Appendix D.1.1 Methodology for additional

details.

Table 6-2. PY10 Efficient Equipment Program (Lighting) Gross Impact Evaluation Sample Design

Stratum Participants (1) Assumed Proportion or Cv in Sample Design

Achieved Sample Size


Prescriptive and Direct Discount Lighting

960 90/10

(Minimum sample size of 10/stratum)

15 Desk Audit

25 Site visit

Program Total 40

(1) A participant is defined as a unique job completed for a unique customer.

6.2.2 Gross Impact Evaluation Results - Lighting

Table 6-3 shows the program’s verified gross energy savings.

Table 6-3. Efficient Equipment (Lighting) Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 67,246 112,402 93,138 272,786 (1)


In PY10, the lighting portion of the Efficient Equipment Program reported energy savings of 96,138

MWh/yr, as shown in Table 6-4, and demand reduction of 14 MW/yr, as shown in Table 6-5. See

Appendix D.1.3 Site Visit and Desk Audit Findings – Lighting for additional information.


Table 6-4. PY10 Efficient Equipment Program (Lighting) Gross Impact Results for Energy

Substratum PYRTD

MWh/yr

Energy Realization

Rate



90% C.L.

PYVTD (MWh/yr) (1)

Lighting -- Small 11,525 92% 0.12 7.08% 10,625

Lighting -- Medium 22,308 91% 0.37 20.18% 20,371

Lighting -- Large 33,404 103% 0.13 8.09% 34,374

Lighting -- Threshold 28,901 96% 0.00 0.00% 27,768

Lighting Total (2) 96,138 97% N/A 4.72% 93,138 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) May not match due to rounding.

Table 6-5. PY10 Efficient Equipment Program (Lighting) Gross Impact Results for Demand

Substratum PYRTD MW/yr

Demand Realization

Rate



90% C.L.

PYVTD (MW/yr) (1)

Lighting -- Small 1.81 99% 0.03 1.49% 1.79

Lighting -- Medium 2.75 98% 0.06 3.47% 2.71

Lighting -- Large 5.24 108% 0.34 20.88% 5.64

Lighting -- Threshold 4.38 87% 0.00 0.00% 3.80

Lighting Total (2) 14.18 98% N/A 7.21% 13.95 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) May not match due to rounding.

Lighting projects achieved 93,138 MWh per year of verified energy savings with a 97% energy realization

rate. Lighting projects achieved 13.95 MW/yr of verified demand reduction with a 98% demand

realization rate. The primary contributors to the energy and demand realization rates that were less

than 100% were differences in verified existing and/or installed fixture quantities, types and wattages,

verified custom lighting fixture hours of use, lighting control types, and verified space conditioning

types. Table D-7 and Table D-8 in Appendix D.3.2 Net-to-Gross Ratio Findings summarize results of the

site visits for the lighting projects.

6.3 Net Impact Evaluation – Lighting

The methods used to determine net savings for downstream, upstream, and midstream programs are

provided in the Evaluation Framework,22 which discusses the common methods used to determine free

ridership and spillover. Cadmus used self-report surveys, administered online and by phone, to assess

free ridership and spillover for the Efficient Equipment Program.

22 Pennsylvania Public Utility Commission. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, and Demand Side Analytics, LLC. Final version May 8, 2018.


Cadmus calculated net savings only to inform future program planning. Energy savings and demand

reduction compliance targets were met using verified gross savings.

Table 6-6 lists the methods and sampling strategy used to determine net savings for the lighting portion

of the Efficient Equipment Program in PY10. Additional details about methodology are in Appendix D.3

Net Impact Evaluation – Lighting and Equipment and Appendix Q Net Impact Evaluation.

Table 6-6. PY10 Efficient Equipment Program (Lighting) Net Impact Evaluation Sample Design

Stratum Stratum Boundaries Population Size Achieved Sample Size NTG Activity

Lighting (1) Participants 960(2) 61 Self-report survey

Program Total 960 61

(1) Prescriptive lighting and direct discount lighting combined. (2) Combined population of prescriptive lighting and direct discount lighting participants.

Table 6-7 shows the free ridership, spillover, and NTG ratios by program stratum.

Table 6-7. PY10 Efficient Equipment Program (Lighting) Net Impact Evaluation Results

Stratum Number of

Surveys

Free Ridership

(%)

Spillover (%)

NTG Ratio Relative Precision

at 90% C.L.

Ex Post kWh/yr Gross Population

Savings

Lighting 61 23% (1) 0% 0.77 12% 93,137,879

Lighting Total 61 23% 0% 0.77 12% 93,137,879

(1) Weighted by the survey sample-verified program kWh/yr savings. This method ensures that respondents who achieved higher energy savings through the program products have a greater influence on the equipment-level free ridership estimate than do the respondents who achieved lower energy savings.

The Phase III Evaluation Framework requires the identification and oversampling of high-impact

measures (HIM) and services to assess free ridership with greater certainty.23 In the Efficient Equipment

Program, Cadmus determined that commercial lighting projects contributed greater than 5% of the

overall PY10 savings to the Non-Residential sector and classified commercial lighting as a high-impact

measure. For net savings calculations, Cadmus attempted to survey all eligible lighting participants and

61 completed the self-report surveys. The relative precision of the high-impact measure NTG ratio

estimate is ±12% precision at 90% confidence.

6.4 Verified Savings Estimates – Lighting

In Table 6-8, the realization rates determined by Cadmus are applied to the reported energy and

demand savings estimates to calculate the verified savings estimates for the lighting portion of the

Efficient Equipment Program in PY10.



Table 6-8. PYTD and P3TD Efficient Equipment Program (Lighting) Savings Summary

Savings Type Energy (MWh/yr) (1) Total Demand (MW/yr) (1)

PYRTD Gross 96,138 14.18


PYVTD Net⁽2⁾ 71,716 10.74

P3RTD Gross 279,668 39.12

P3VTD Gross 272,786 39.58

P3VTD Net (2) 201,053 29.19 (1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.

6.5 Participation and Reported Savings by Customer Segment – Equipment

6.5.1 Definition of an Equipment Participant

An equipment participant is defined as a unique job initiated by a unique customer. A unique customer

can submit multiple equipment jobs in different equipment categories (HVAC, refrigeration, motors,

food service, office, and agricultural). In PY10, the equipment portion of this program had 72 equipment

jobs and 68 unique equipment customers (149 database records). All but 10 of the PY10 equipment jobs

followed the prescriptive delivery channel. Cadmus grouped the 10 direct discount delivery channel

equipment jobs with the prescriptive stratum in PY10 because the population in this delivery channel

was so small.

6.5.2 Program Participation and Reported Impacts for Equipment


payments for the equipment portion of Efficient Equipment Program in PY10, by customer segment.

Table 6-9. PY10 Efficient Equipment Program (Equipment) Participation and Reported Impacts


PYTD # Participants (2) 6 7 0 59 72

PYRTD MWh/yr 230 836 0 2,706 3,772

PYRTD MW/yr 0.01 0.06 0.00 0.21 0.27

PYVTD MWh/yr 202 583 0 2,274 3,059

PYVTD MW/yr 0.01 0.04 0.00 0.19 0.23

PY10 Incentives ($1000) (3) N/A (1) May not match due to rounding. (2) Participants are defined as a unique job initiated by a unique customer by equipment categories. (3) Incentives are tracked at the program level.


6.6 Gross Impact Evaluation – Equipment

6.6.1 Gross Impact Evaluation Activities - Equipment

The evaluation sampling strategy is shown in Table 6-10. See Appendix D.2.1 Methodology for additional

details.

Table 6-10. PY10 Efficient Equipment Program (Equipment) Gross Impact Evaluation Sample Design

Stratum Participation

(Unique Jobs)

Assumed Proportion or Cv in Sample Design



Prescriptive and Direct Discount Equipment

72 85/15 2 Desk audit

17 Site visit

Total 72 85/15 19

6.6.2 Gross Impact Evaluation Results – Equipment


Table 6-11. Efficient Equipment (Equipment) Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 3,671 3,592 3,059 10,322 (1)


In PY10, the equipment portion of the Efficient Equipment Program reported energy savings of

3,772 MWh/yr, as shown in Table 6-12, and demand reduction of 0.27 MW, as shown in Table 6-13.

Table 6-12. PY10 Efficient Equipment Program (Equipment) Gross Impact Results for Energy

Substratum PYRTD

MWh/yr

Energy Realization

Rate



85% C.L.

PYVTD (MWh/yr) (1)

HVAC 253 87% 0.01 2.39% 219

HVAC – Occupancy Sensors 674 31% 1.27 131.87% 208

Motors 881 88% 0.21 13.74% 773

Other 29 22% 2.21 459.66% 6

Refrigeration 1,935 96% 0.14 7.22% 1,852

Total (2) 3,772 81% N/A 7.82% 3,059 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

(2) May not match due to rounding.


Table 6-13. PY10 Efficient Equipment Program (Equipment) Gross Impact Results for Demand

Substratum PYRTD MW/yr Demand

Realization Rate



85% C.L.

PYVTD (MW/yr) (1)

HVAC 0.08 129% 0.01 1.74% 0.11

HVAC – Occupancy Sensors 0.06 31% 1.13 117.38% 0.02

Motors 0.06 92% 0.29 19.17% 0.05

Other (2) 0.01 24% 2.14 446.00% 0.00

Refrigeration 0.06 85% 0.47 24.57% 0.05

Total (3) 0.27 87% N/A 9.12% 0.23 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings

(2) The Other substratum includes agricultural projects. (3) May not match due to rounding.

Equipment projects achieved 3,059 MWh per year of verified energy savings with an 81% energy

realization rate. Equipment projects achieved 0.23 kW/yr of verified demand reduction with an 87%

demand realization rate. The primary contributors to the energy and demand realization rates that were

different from 100% were in-service rates, incorrect reported equipment capacities, and incorrect

baseline classification. Additional information is in Appendix D.2 Gross Impact Evaluation – Equipment.

6.7 Net Impact Evaluation – Equipment

Table 6-14 lists the methods and sampling strategy used to determine net savings for the equipment

portion of the Efficient Equipment component of the Non-Residential Energy Efficiency Program in PY10.

Additional details about methodology are in Appendix D.3 Net Impact Evaluation – Lighting and

Equipment and Appendix Q Net Impact Evaluation.

Table 6-14. PY10 Efficient Equipment Program (Equipment) Net Impact Evaluation Sample Design

Stratum Stratum

Boundaries Population

Size Achieved Sample

Size NTG Activity

Prescriptive and Direct Discount Equipment Participants 72 7 Self-report survey

Total 72 7

For net savings calculations, Cadmus attempted to survey all eligible equipment participants and seven

completed the self-report surveys. Table 6-15 shows the free ridership, spillover, and NTG ratios by

program stratum.


Table 6-15. PY10 Efficient Equipment Program (Equipment) Net Impact Evaluation Results

Stratum Number of

Surveys

Free Ridership

(%)

Spillover (%)


at 85% C.L.

Ex Post kWh/yr Gross Population

Savings

Prescriptive and Direct Discount Equipment

7 36% (1) 0% 0.64 46% 3,058,840

Total 36% 0% 0.64 46% 3,058,840

(1) Weighted by the survey sample-verified program kWh/yr savings. This method ensures that respondents who achieved higher energy savings through the program products have a greater influence on the equipment-level free ridership estimate than do the respondents who achieved lower energy savings.

6.8 Verified Savings Estimates – Equipment


demand savings estimates to calculate the verified savings estimates for the equipment portion of the

Efficient Equipment Program in PY10.

Table 6-16. PYTD and P3TD Efficient Equipment Program (Equipment) Savings Summary




PYVTD Net (2) 1,958 0.15

P3RTD Gross 12,439 0.99

P3VTD Gross 10,322 0.83

P3VTD Net ⁽2) 6,832 0.55 (1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.

6.9 Process Evaluation – Lighting and Equipment

6.9.1 Research Objectives

The main research objectives for the PY10 evaluation of the Efficient Equipment Program focused on

customer experience, program performance, and program influence.

6.9.2 Evaluation Activities

The PY10 process evaluation for the Efficient Equipment Program involved these research activities:

• Interviews with PPL Electric Utilities and ICSP program managers

• Online and telephone participant surveys

• Participation was low in the equipment stratum, so Cadmus did not reach the targeted number

of completed surveys in this stratum (23), and did not reach the overall target of 69 completed

surveys.


Table 6-17 lists the process evaluation sampling strategy for the lighting and equipment rebates.

Additional details about sampling methodology are in Appendix D.4.2 Survey Approach and Appendix R

Survey Methodology.

Table 6-17. Process Evaluation Sampling Strategy for the Efficient Equipment Program

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion

or Cv in Sample Design

Target Sample

Size

Achieved Sample Size(3)

Records Selected

for Sample

Frame (1)

Percent of Sample Frame Contacted to

Achieve Sample

(2)

Equipment and Lighting

PPL Electric Utilities Program and ICSP Staff

Staff Telephone in-depth interview

2 N/A 2 2 N/A 100%

Participants

Equipment

Online survey

72

0.5

69

4

33 100% Telephone survey

0.5 3

Prescriptive lighting

Online survey

707 0.5

10


27

Direct discount

Online survey

253 0.5

15


9

Program Total

1,034 71 70 425

(1) Sample frame is a list of participants with contact information who had a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities database. After selecting all unique records, Cadmus removed any records from the population that had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (email or telephone number), were on the do not call list, or opted out of the online survey. See Appendix D.4.2 Survey Approach. (2) Percent contacted means the percentage of the sample frame contacted to complete surveys. (3) Cadmus attempted to complete 23 surveys in each of the three strata but was unable to do this in the equipment stratum due to lack of available records. All available equipment records were exhausted in an attempt to reach the equipment stratum target.

6.9.2.1 Survey Methodology

Cadmus conducted online and telephone surveys with 68 participants of the Efficient Equipment

Program using a stratified random sample. Twenty-nine participants responded to the online survey and

39 to the telephone survey between February 2019 and July 2019. These surveys asked identical

questions to assess program satisfaction, net savings, and the influence of the program and of the

contractor or design engineer on project design, purchase decision, and program participation.

Additional information about survey methodology can be found in Appendix R Survey Methodology.


6.9.2.2 Program Staff and ICSP Interviews

In March 2019, Cadmus conducted interviews with the program managers from PPL Electric Utilities and

the ICSP. The interviews focused on identifying and assessing changes to program design and delivery

from PY9 to PY10 and understanding the areas working well and any possible challenges.

6.9.3 Process Evaluation Findings

6.9.3.1 Program Delivery

The Efficient Equipment Program was delivered effectively in PY10 and maintains high levels of

customer satisfaction. The ICSP delivered the PY10 Program the same as in PY9, with three primary

exceptions. PPL Electric Utilities initially accepted applications for the VFD compressors in PY9; however,

they are now a part of the Custom Program. PPL Electric Utilities has also added refrigeration projects to

the Direct Discount delivery channel in PY10. Lastly, PPL Electric Utilities launched an email campaign in

January 2019 to promote Midstream and Direct Discount rebates.

6.9.3.2 Program Satisfaction

Of the 68 participants who responded to the survey, 91% of participants were either very satisfied with

the program overall (75%) or somewhat satisfied (16%). Though this was a decrease from PY9,24 where

overall satisfaction was 95% (n=65), it was not a significant change. Figure 6-1 shows the satisfaction

results by the three program components: prescriptive lighting, direct discount lighting, and equipment

participants.

Figure 6-1. PY10 Participant Satisfaction

Source: Survey question, “Now, thinking about your overall experience with PPL Electric Utilities' business

energy-efficiency program, how would you rate your satisfaction?”

Prescriptive Lighting includes one response from a partially completed survey.

24 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to PA PUC. Prepared by Cadmus. November 15, 2018.


Two equipment respondents and three prescriptive lighting respondents were not too satisfied with the

program overall. Table 6-18 shows these respondents’ comments and suggested improvements.

Table 6-18. Comments and Suggested Improvements for the Efficient Equipment Program Overall

Stratum Improvements

Prescriptive Lighting (n=3)

“The amount of time it takes for the process and [the] number of hoops to go through are absolutely ridiculous. Maybe PPL [Electric Utilities] set it up that way to discourage participation?”

“The processing time to get the rebate check was way too long. It was also very difficult to get information in regards to the rebate check to help us track it down on our end.”

“[I] was led to believe the rebates would be much higher.”

Equipment (n=2)

“We never got our rebate - [we needed] clearer information.”

“Make the interface easier to navigate - we don't always have all of the information available to use. Nailing down all of the data and information is hard and sometimes not worth it for the amount of the rebate. [PPL Electric Utilities’] needs a better way of collecting data or have someone come out and do it for us. [This] will help us feel more confident about our project.”

Source: Survey question, “What is the one thing PPL Electric Utilities or CLEAResult could change about the program to improve it?”

Figure 6-2 shows participant satisfaction with elements of the Efficient Equipment Program.

Respondents were most satisfied with the professionalism of the program representatives (73%);

however, this was a significant decrease in satisfaction from PY9, 25,26 where 96% (n=51) of respondents

were very satisfied with the professionalism of the program representatives. Though these percentages

are very small, respondents were least satisfied with the information provided about the application

process (4% not too satisfied and 2% not at all satisfied; n=54). Two equipment (n=7) and one direct

discount lighting (n=19) respondent were not too satisfied or not at all satisfied with information

provided about the application process.

25 Statistically significant at the 5% level.



Figure 6-2. Participant Satisfaction with Different Program Elements

Source: Survey Question “Please indicate how satisfied you are with each one.”

More than a third of respondents (38%; 26 of 68) provided recommendations to improve the program.

Table 6-19 shows the suggested improvements.

Table 6-19. Suggested Improvements for Elements of the Efficient Equipment Program

Suggested Improvement Percentage of Responses (n=26)

Provide clearer information/communication about the program 38%

Simplify application process 19%

Increase rebate amount or types of rebates 19%

Simplify kWh savings calculator 12%

Reduce rebate processing time 8%

Issue rebates directly to the customer 4%

Source: Survey question, “What is the one thing PPL Electric Utilities or CLEAResult could change about the program to improve it?” (n=26).

Areas Working Well

The survey provided a list of items and asked respondents to select which were working well.

Respondents thought the rebates they received, communication with PPL Electric Utilities staff, and the

time it took to receive the rebate were the top three program elements that worked well (Figure 6-3).


Figure 6-3. PY10 Areas Working Well in the Efficient Equipment Program

Source: Question, "Thinking about what worked well with the business energy-efficiency program,

what one item worked best? What worked next best?" (n=68) Multiple responses allowed.

Program Influence

The survey asked respondents how much influence the program components had on their decision to

complete the project in the way they did (Figure 6-4). In PY10, the energy services company (ESCO),

contractor, vendor, or consultant who helped design the project were the most influential, with an

average score of 4.14 (n=57). Direct Discount lighting program respondents reported PPL Electric

Utilities’ rebates and PPL Electric Utilities’ information about energy efficiency as extremely influential at

a significantly higher rate than did prescriptive lighting respondents.27 Additionally, Direct Discount

program respondents reported that the ESCO, contractor, vendor, or consultant who helped them

design their project as extremely influential at a significantly higher rate than did prescriptive lighting

respondents.28

27 Statistically significant at the 5% level. Direct Discount respondents reported an average score of 4.48 (n=23) and Prescriptive Lighting respondents reported an average score of 4.11 (n=37).

28 Statistically significant at the 10% level. Direct Discount respondents reported an average score of 4.30 (n=23) and Prescriptive Lighting respondents reported an average score of 3.41 (n=37).


Figure 6-4. Participants’ Average Influence Scores

Source: Survey question “Please rate each item on how much influence it had on the decision to complete the project

the way it was completed on a scale from 1 to 5 where 5 is extremely influential and 1 is no influence.”


Because the Efficient Equipment component is part of the Non-Residential Energy Efficiency Program,

cost-effectiveness is presented in section 5.4 Cost-Effectiveness Reporting.


6.11 Recommendations – Lighting and Equipment

Overall, the Efficient Equipment Program has been successful, with the verified savings of 96,197 MWh/year. Most survey respondents

(91%; n=68) were satisfied with the program. Recommendations are provided in Table 6-20, along with a summary of how PPL Electric Utilities

plans to address the recommendations.

Conclusion 1: The application process is challenging for a

small number of survey respondents and the clarity of

program information and application requirements were of

concern. Both aspects led to these respondents’ decreased

satisfaction. While the program provides instruction about

how to complete the application and offers customers a

way to track rebate status, findings suggest that additional

information is needed to improve customers’

understanding and satisfaction with the rebate application

process.

Findings Support

• The majority, 91%, of survey participants (n=68) were very or somewhat satisfied with

the program.

• Given a list of program components, four were least satisfied with the information

provided about the application process and recommended improvements to the

application process.

• 38% of respondents (n=26) recommended improving communication about the

program. (See section 6.9.3.2 Satisfaction with Elements of the Efficient Equipment

Program.)

Conclusion 2: Contractors, vendors, and distributors are

important and influential to the program’s delivery.

Findings Support

• Almost half of the survey respondents (48%; n=68) participated in the Efficient

Equipment Program prior to PY10, and of these 33 respondents, 56%, said they

worked with a contractor, vendor, or distributor for their project. (See Appendix

D.4.1 Additional Findings.)

• Of the respondents who participated in the Efficient Equipment Program for the

first time in PY10 (n=30), 90% said they worked with a contractor, vendor, or

distributor for their project. (See Appendix D.4.1 Additional Findings.)

• Additionally, 57% of respondents said they did not know about the PPL Electric

Utilities’ rebate before interacting with their ESCO, contractor, vendor or other

consultant (n=68). (See Appendix D.4.1 Additional Findings.)


Conclusion 3: Third parties such as energy services

companies (ESCO), contractors, vendors, and consultants

have had a large role in the way customers and contractors

design and implement equipment and lighting projects.

Findings Support

• Energy services company (ESCO), contractors, vendors, and consultants were quite

influential in designing and implementing customers’ projects. (rated 4.14 out of 5,

where 5 is extremely influential). This was especially true of Direct Discount lighting

participants, but not surprising since that is the way this program delivery channel

is designed. (See section 6.9.3.2 Program Satisfaction.)

Table 6-20. Status of Recommendations for the Efficient Equipment Program

Efficient Equipment Program

Conclusion Recommendation EDC Status of Recommendation

(Implemented, Being Considered, Rejected and Explanation of Action Taken by EDC)

Conclusion 1: The application process is challenging for a small

number of survey respondents and the clarity of program information

and application requirements were of concern. Both of these aspects

led to these respondents’ decreased satisfaction. While the program

provides instruction about how to complete the application and offers

customers a way to track rebate status, findings suggest that

additional information is needed to improve customers’

understanding and satisfaction with the rebate application process.

Consider displaying already developed tools and tips (e.g., FAQ page, PPL Electric Utilities’ contact information, and other tools) more prominently on the home screen of the Non-Residential Program website. This may help improve communication by making these essential tools more readily visible and available to customers.

Implemented.

Conclusion 2: Contractors, vendors, and distributors are important

and influential to the program’s delivery. Consider providing program contractors and other third party companies with leave-behind materials for customers such as PPL Electric Utilities’ contact information, how to access the rebate tracking portal, how rebate amounts are calculated, and information about all Non-Residential rebate programs.

Being considered. Conclusion 3: Third parties such as energy services companies (ESCO),

contractors, vendors, and consultants have had a large role in the way

customers and contractors design and implement equipment and

lighting projects.

Midstream Lighting is designed to make choosing and procuring high-efficiency lighting from a participating lighting distributor simple and fast, by discounting qualifying LED lamps, bulbs, and fixtures at the point of sale.

MIDSTREAM LIGHTING



PY10 saved

42,137 MWh/yr

24,306 MWh/yr

9%of Non-Residential program savings




distributors satisfied with program92%

contractors satisfied with program100%

end users satisfied with program100%

PY10 PARTICIPATION

19 Distributors participated

7,633 Jobs (sales)

3,256 Unique PPL Electric Utilities customers

Chapter 7 Non-Residential Midstream Lighting Program PPL Electric Utilities | 66

7 Non-Residential Midstream Lighting Program

The Midstream Lighting component of the Non-Residential Energy Efficiency Program is designed to

make choosing and procuring high-efficiency lighting simpler and faster than typical downstream

program delivery channels. Contractors and PPL Electric Utilities customers may purchase qualifying LED

lamps, bulbs, and fixtures directly from a participating lighting distributor. The purchaser receives an

instant discount through a discounted list price at the point of sale. PPL Electric Utilities pays the

distributor the discount, and the distributor is required to pass this discount along to the purchaser.

7.1 Participation and Reported Savings by Customer Segment

7.1.1 Definition of a Participant

Distributors, typically an electric equipment supply outlet, are considered the participants in the

Midstream Lighting Program because they receive the incentives. The primary impact evaluation

sampling unit in Midstream Lighting is a unique job. A job is a participating distributor’s sale of qualified

products to a specific business at a specific point in time.

7.1.2 Program Participation and Reported Impacts


payments for all the components of the Midstream Lighting Program in Phase III, by customer segment.

Table 7-1. PY10 Midstream Lighting Program Participation and Reported Impacts


PYTD # Participants 2,110 801 251 4,471 7,633

PYRTD MWh/yr 7,086 3,485 409 12,563 23,542

PYRTD MW/yr 1.37 0.63 0.08 2.38 4.46

PYVTD MWh/yr 7,201 3,689 426 12,989 24,306

PYVTD MW/yr 1.31 0.61 0.07 2.28 4.27

(1) Total may not sum to all columns due to rounding.


7.2.1 Gross Impact Evaluation Activities

7.2.1.1 Data Collection

Cadmus obtained the ICSP’s records for jobs in the evaluation sample, consisting of the ICSP’s rebate

reports, distributors’ invoices submitted to the ICSP, and the distributors’ records of sales to the

contractor or end user. The ICSP also provided their QA/QC documentation, if QA/QC had been

completed for sampled jobs.


7.2.1.2 Sample Design

Cadmus sampled Midstream Lighting jobs to estimate the realization rate and verified savings with ±15%

precision at 85% confidence, assuming a coefficient of variation of 0.7. In PY10, the program reported

7,633 jobs in PPL Electric Utilities tracking database. Cadmus drew a sample from each of the first three

quarters from the population of Midstream Lighting jobs using probability proportional to size sampling,

so that the largest jobs in the program had the largest probability of selection, but all jobs were eligible

and had some probability of being selected. This ensured that a robust sample of jobs was used to

estimate the program total savings and realization rates. There were no threshold lighting jobs in the

PY10 Midstream population.

Cadmus defined a site as a business located at a given address. After completing a site visit to verify a

randomly sampled job, Cadmus verified additional jobs installed at that site during PY10 as well. Cadmus

referred to these jobs as siblings to the randomly sampled jobs and assigned them to the convenience

stratum. For the evaluation of this program, sibling job verification results were included in the

calculation of realization rates but not in the calculation of evaluation relative precision, which is based

solely on the random sample of jobs.

Cadmus conducted 14 site visits and 13 desk audits to evaluate 27 randomly sampled jobs and 60

siblings in PY10, for a total of 87 sampled job verifications. One additional job previously sampled in PY9

remained unverified in PY9. Cadmus verified this job during the PY10 verification effort.

The program sample sizes are shown in Table 7-2. See Appendix E.1 Gross Impact Evaluation for

additional details about methodology.

Table 7-2. PY10 Midstream Lighting Program Gross Impact Evaluation Sample Size

Stratum Population Size (Jobs)




Midstream Lighting PY10 Random Sample

7,730 (1) 0.7

27 Records review, and either desk audit or site visit Midstream Lighting PY10

Convenience Sample 60

Program Total 7,730 0.7 87 (1) In PY10, there were 8,421 records in PPL Electric Utilities’ tracking database corresponding to 7,633 unique jobs for Midstream Lighting. Jobs in PPL Electric Utilities’ database that included high bay/low bay as well as other lighting products in the same job were treated as two jobs and stratified separately. That is, the job was separated into high bay/low bay products and other products. Therefore, the total population of jobs across all strata for the impact evaluation is 7,730.

For the sampled jobs, Cadmus conducted a review of records, as well as either site visits or desk audits.

Table 7-3 shows the number of verifications completed using a desk audit or a site visit. See Appendix

E.1 Gross Impact Evaluation for details on these activities.


Table 7-3. PY10 Midstream Lighting Impact Evaluation Activities

Evaluation Activity Randomly

Sampled Jobs Convenience Sampled Jobs

(Siblings)

Total Sampled

Jobs

Notes

Records Review 27 60 87 -

Desk Audit (phone verification) 13 - 13 Nested within Records Review Sample

Site Visit 14 60 74 Nested within Records Review Sample

7.2.2 Gross Impact Evaluation Results

In PY10, Midstream Lighting reported energy savings of 23,542 MWh/yr and demand reduction of 4.46

MW/yr. Table 7-4 highlights the growth in verified savings, demand reduction, number of jobs,

distributor participants, and customers from PY8 through PY10.

Table 7-4.Midstream Lighting Program PY8 through PY10 Participation Growth

Year Distributors Jobs(1) Unique Customers PYVTD MWh/yr PYVTD MW/yr

PY8 12 789 437 1,917 0.34

PY9 17 4,685 2,046 15,915 2.74

PY10 19 7,633 3,256 24,306 4.27

[1] A job is a participating distributor’s sale of qualified products to a specific business at a specific point in time.

The program achieved 103% and 96% realization rates for energy savings and demand reduction

respectively, as shown in Table 7-5 and Table 7-6, at a relative precision of ±10.74%.This is better than

the target of ±15% precision for the program evaluation. In most strata, the reported energy realization

rates were between 85% and 125%, with stratum error ratios less than 0.5. The convenience stratum,

however, had a high error ratio and high realization rate. The error ratio was high in the convenience

stratum because this stratum included projects of all sizes. The realization rate in this stratum was high

due to a few large projects with significantly higher verified savings than reported savings resulting from

multiple adjustments, such as hours of use and baseline wattage.

Table 7-5. PY10 Midstream Lighting Program Gross Impact Results for Energy

Stratum PYRTD

MWh/yr

Energy Realization

Rate

Sample Cv or Error

Ratio


85% C.L.

PYVTD MWh/yr (1)

Midstream Lighting – Convenience Sample(2) 413 209% 0.95 N/A 863

Midstream Lighting -- High Bay/Low Bay 4,613 94% 0.50 39.61% 4,314

Midstream Lighting – Large 1,044 106% 0.40 95.45% 1,105

Midstream Lighting – Medium 5,928 98% 0.27 35.78% 5,792

Midstream Lighting – Medium-Large 5,092 85% 0.37 25.38% 4,307

Midstream Lighting – Small 6,451 123% 0.33 15.55% 7,925

Midstream Lighting Total(3) 23,542 103% N/A 10.74% 24,306 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) The convenience sample is included in the calculation of realization rates but not included in the calculation of evaluation relative precision.



Table 7-6. PY10 Midstream Lighting Program Gross Impact Results for Demand

Stratum PYRTD MW/yr

Demand Realization

Rate

Sample Cv or Error

Ratio

Relative Precision

at 85% C.L.

PYVTD MW/yr (1)

Midstream Lighting – Convenience Sample (2) 0.07 159% 0.66 N/A 0.12

Midstream Lighting -- High Bay/Low Bay 0.94 74% 0.39 30.94% 0.70

Midstream Lighting – Large 0.15 93% 1.09 262.10% 0.13

Midstream Lighting – Medium 1.11 95% 0.34 44.51% 1.06

Midstream Lighting – Medium-Large 1.00 79% 0.45 31.10% 0.79

Midstream Lighting – Small 1.19 123% 0.24 11.33% 1.47

Midstream Lighting Total (3) 4.46 96% N/A 11.18% 4.27 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) The convenience sample is included in the calculation of realization rates but is not included in the calculation of evaluation relative precision.


In the Midstream Lighting Program, neither the distributor nor the customer is required to complete a

PA TRM Appendix C lighting savings calculator for the job. The distributor must report each sale and

include information about the product, the product quantity, the purchaser, and the address of the

intended installation. However, key variables required to determine savings for the job, such as facility

type and space conditioning type, are often unknown by the ICSP.

The ICSP assigns the facility type, and ex ante baseline and efficient fixture types to qualified products as

prescribed in the Midstream Lighting 2016 PA TRM – Interim Measure Protocol (IMP).29 The reported

savings are computed assuming a 98% installation rate, according to the IMP.

Cadmus adjusted these key reported variables based on its verification activities. Cadmus made

adjustments where applicable IMP prescriptive inputs were not used by the ICSP, where the verified

variables differed from those assumed by the IMP (e.g., in-service rate), or where the verified variables

differed from those assigned by the ICSP (e.g., facility type).

The most frequent discrepancies between reported and verified variables were the facility type, which,

in turn, determine the hours of use and coincidence factors specified by the IMP and the space

conditioning type. See Appendix E.1.2 Verification Findings for details on the verification findings.


Cadmus applied the NTG ratio calculated in PY9 for the program in PY10. In PY9, Cadmus used self-

report surveys to assess free ridership for Midstream Lighting. Cadmus followed the methodology used

29 Pennsylvania Public Utility Commission. 2016 PA TRM – Interim Measure Protocol: Lighting Improvements for Midstream Delivery Programs. Version approved January 2019, effective of June 1, 2018–May 31, 2020.


to determine net savings for midstream programs in the Evaluation Framework,30 which discusses the

common methods to determine free ridership and spillover.

The NTG ratio determined in PY9 and applied to PY10 is 0.85. Cadmus maintained the PY9 NTG ratio

because it was not substantially different from the 0.86 ratio evaluated in PY8. In PY10, PPL Electric

Utilities adjusted (reduced) incentives somewhat, but only to keep in line with lower retail prices. There

were no substantive program changes in PY10. Therefore, Cadmus did not expect free ridership to be

different in PY10. Details about the PY9 NTG ratio methodology and results are in the PY9 Annual

Report.31



demand savings estimates to calculate the verified savings estimates for Midstream Lighting in PY10.

Table 7-7. PYTD and P3TD Midstream Lighting Program Savings Summary


PYRTD 23,542 4.46


PYVTD Net⁽²⁾ 20,660 3.63

P3RTD 46,078 8.60

P3VTD Gross⁽3⁾ 42,137 7.35

P3VTD Net⁽², 3⁾ 35,836 6.25 (1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.

(3) PY9 savings that were unverified in PY9 were verified in PY10 as 0 kWh/yr.



The process evaluation of the Midstream Lighting Program focused on these main research objectives:

• Assess customer and distributor satisfaction and experience

• Emphasize areas of program success and challenges

• Assess the influence of both distributor and instant discounts on purchaser decisions

• Measure the influence of the program on contractors’ business practices and sales


31 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to PA PUC. Prepared by Cadmus. November 15, 2018. http://www.puc.pa.gov/pcdocs/1595564.pdf



• Identify motivators for and barriers to participation in Midstream Lighting

• Track market progress indicators (MPIs) including perceptions of the cost-effectiveness of

efficient lighting products


The PY10 process evaluation activities for the Midstream Lighting Program included these:

• Conduct interviews with PPL Electric

Utilities and ICSP program managers

• Conduct telephone interviews with

participating distributors

• Conduct telephone interviews with purchasers,

including end users and contractors

• Review PPL Electric Utilities’ tracking database

and the ICSP’s tracking data

The process evaluation activities were consistent with the evaluation plan, with one exception. Cadmus

did not interview end users who were not purchasers (i.e., those for whom a contractor or another third

party purchased lighting). The objectives for interviewing these end-user nonpurchasers were primarily

to collect data for the net impact evaluation, which Cadmus did not repeat in PY10 (see section 7.3 Net

Impact Evaluation).

Cadmus split the target sample of purchasers into two groups—end-user purchasers and contractor

purchasers—and completed seven interviews with each group. This was a sufficient sample to collect

qualitative data to support the process and market effects evaluations.

Cadmus attempted to reach and interview all 19 participating distributors, with the expectation of

completing at least 15 interviews. Cadmus completed 13 interviews because, even after multiple

attempts, six distributors could not be reached to schedule and complete interviews.

Table 7-8 lists the process evaluation sampling strategy. Additional details about sampling methodology

are included in Appendix F Evaluation Detail – Custom Program and Appendix R Survey Methodology.


Table 7-8. Process Evaluation Sampling Strategy for Midstream Lighting

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size

Achieved Sample

Size

Records Selected

for Sample

Frame (1)

Percent of Sample Frame

Contacted to Achieve Sample (2)



2 N/A 2 2 2 100%

Participating Distributors

Distributors Telephone in-depth interview

19(4) N/A 15 13 19 100%

End-User Purchasers

End-user customers who purchase directly from distributors

Telephone in-depth interview

6,214(3) N/A 7 7 1,048 3%

Contractor Purchasers

Contractor purchaser for whom contact data were provided


6,214(3) N/A 7 7 314 6%

Program Total 31 29 (1) Sample frame is a list of participants with contact information who have a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities database. After selecting all unique records, Cadmus removed any duplicate records from the population. (2) Percent contacted means the percentage of the sample frame contacted to complete surveys. (3) Contractors and End-User Purchasers started from the same population list. (4) Cadmus counted distributors with multiple locations, or those that are subsidiaries of the same parent company, as one, for purposes of interview sampling.

7.5.2.1 Participating Distributor Interview Methodology

In PY10, 19 unique distributors participated in the Midstream Lighting Program. Cadmus attempted to

contact all of them and completed in-depth interviews with 13, who represented approximately 84% of

reported PY10 unit sales. Five of these 13 distributors were interviewed for the first time in PY10 (two

were new to the program in PY10). Cadmus asked these first-time interviewees about stocking practices

and the estimated proportion of sales of efficient versus standard-efficiency products before and after

their participation. Cadmus asked the eight distributors who had been interviewed in PY9 about sales in

PY10 and any observed changes over the past year.


7.5.2.2 Purchaser and End User Interview Methodology

Cadmus conducted interviews with seven contractor purchasers and seven end-user purchasers.

Cadmus selected a sample from each group, prioritized by job size, from the PY10 ICSP tracking data

where the contractor name was available.32


This section presents the findings of the process evaluation. Additional detail regarding the process

evaluation methodology and findings can be found in Appendix E.2 Process Evaluation for Midstream

Lighting.


In PY10, Midstream Lighting sales increased 24% (in quantity of products) over sales in PY9. Two of the

19 distributors reported sales in PY10 for the first time.

Updates to Reporting System

Midway through PY10 (January 1, 2019), the ICSP activated a portal-based system for distributors, which

they used to validate qualifying products and report sales. All distributors reported that this was an

improvement over the prior system (once they learned how to use it) for several reasons. The system

helps track sales, eliminates the need for each distributor to maintain and update a Qualifying Products

List (QPL), makes it easier to track the eligibility of both customers and products, and makes it easier to

submit invoices.

Distributors provided the following suggestions for improving the portal:

• Allow customer account validation by address (two distributors, discussed below)

• Improve the QPL search functionality (two distributors)

• Improve the template for uploading data to the portal; increase the size of the data fields to

accommodate data (one distributor)

• Add more information to the dashboard about the status of incentive checks (one distributor)

The new portal system requires that distributors enter a valid PPL Electric Utilities account number for

the end user for all sales. This requirement is intended to allow distributors to verify eligibility prior to

making a sale, thus reducing leakage (i.e., the installation of program products outside of PPL Electric

Utilities’ service territory). Most distributors said entering the account number was no more than a

minor inconvenience; nevertheless, during the first few months, this requirement delayed some sales

because customers did not know in advance they would need their account number.

Only two of 13 distributors reported a significant drop in sales after the account number requirement

was introduced. One said the drop in sales was more pronounced with high-volume customers because

32 Cadmus used the purchaser and customer name fields to identify the purchaser type (end-user or contractor) for each job. The ICSP provided the purchaser name for approximately half of PY10 jobs.


the person purchasing lighting did not typically handle the electric bill. Four distributors thought the

requirement was a larger issue for contactors. However, when Cadmus asked contractors, none said

they had an issue getting the account number from customers. Three said they get it during their

standard process of collecting customer information. All seven contractors knew about the requirement

prior to meeting with their customers.

Program Marketing

In PY10, as in PY9, marketing for Midstream Lighting was conducted primarily through the distributor

channel, wherein participating distributors used point-of-purchase materials, provided by the ICSP, to

educate counter sales staff and promote discounted products. The ICSP reported that more direct

marketing to commercial customers was still pending as of PY10.

7.5.3.2 Program Awareness

Cadmus included questions in participant surveys about customers’ awareness of Midstream Lighting

(Table 7-9). Questions were included in the Non-Residential Energy Efficiency Program surveys, including

Prescriptive Lighting, Efficient Equipment, and Custom, as well as in the general small-commercial

population survey. Only 27% of respondents in the downstream rebate programs were aware of

Midstream Lighting (in PY9 34% of downstream program respondents were aware of Midstream

Lighting). Similarly, 25% of the small commercial general population survey respondents were aware of

Midstream Lighting.

Table 7-9. Percentage of Commercial Rebate Customers Aware of Midstream Lighting Program

Program PY8 PY9 PY10

Custom 38% (n=16) 50% (n=24) 45% (n=21)

Prescriptive Lighting 26% (n=61) 23% (n=56) 17% (n=10)

Prescriptive Equipment 2 of 7 5 of 8 4 of 7

General Small C&I Population 19% (n=269) N/A 25% (n=361)

Table 7-10 summarizes how respondents in each of the two purchaser groups interviewed heard about

Midstream Lighting. Consistent with PY9, nearly all of the contractors and end-user purchasers in PY10

learned of the program from a distributor.

Table 7-10. Midstream Lighting Awareness

Participant Group

Participant Group Definition

Primary Way Purchasers Learned about

Midstream Lighting

Knew the Amount of the Discount

Knew about Prescriptive

Rebate Program

Contractors (n=7)

Purchased qualified products for their

customers Distributors (n=6)

Not asked, but all seven contractors either always (n=3)

or often (n=4) incorporate rebate into their bid)

4

End-user purchasers (n=7)

Purchased products for their business

directly from participating distributor

Distributors (n=7)

1 yes 6 no

6


Of the interviewed contractors who also knew of the Efficient Equipment Program’s lighting component,

which offers downstream prescriptive rebates, three said they preferred Midstream Lighting because it

was less work on their end (n=2) and their customers preferred the instant rebate (n=2). Two

contractors said they frequently use both, choosing the most appropriate according to the customer’s

needs and product cost. One said it comes down to getting lighting equipment to customers “in the

easiest and least expensive way,” and the other said “It’s about finding the right solution at the right

price.”


Overall Satisfaction

Satisfaction with Midstream Lighting was relatively high, with the majority of respondents stating they

were very satisfied. In PY10, as in PY9 and PY8, all purchasers interviewed (end-user purchasers and

contractors) said they were either somewhat satisfied or very satisfied with Midstream Lighting. Only

one distributor respondent was not too satisfied with Midstream Lighting in PY10.

In PY10, 86% of all purchasers interviewed (n=7) were very satisfied. This is not significantly different

from PY9,33 when 71% (n=14) of purchasers were very satisfied. The same was true for contractors and

distributors; results not significantly different from PY9 to PY10. Figure 7-1 compares satisfaction results

for PY9 and PY10.

Figure 7-1. PY9 and PY10 Distributor, Contractor, and End-User Satisfaction

Source: Distributor (H8/H9), Contractor (D1), and End-User (D1) survey question: “Thinking about your overall experience with

PPL Electric Utilities’ Midstream Lighting Program, how would you rate your satisfaction? Would you say you are…?”

33 Differences from PY9 are not statistically significant at 90% confidence level.


Thirteen of 14 purchasers interviewed stated they (or their clients, in the case of contractor purchasers)

were satisfied with the energy savings from the lighting products they purchased through Midstream

Lighting. Others stated lighting quality is important, along with non-energy benefits.

Areas Working Well

Overall, 10 of 13 distributors and all seven contractors thought program incentives were sufficient to

encourage customers to purchase program products. Several distributors said the variety of program-

eligible products was sufficient to meet most of their customers’ needs, which is consistent with PY9.

Distributors agreed that the instant discounts help drive sales and that the new portal-based reporting

system was an improvement. As in PY9, most distributors said they participate in Midstream Lighting to

stay competitive and that participating in a utility program adds credibility to selling high-efficiency

products. Most distributors agreed that without the Midstream Lighting incentives, sales of program-

qualified products would be lower, but most could not determine how much lower.

End-user purchasers and contractors appreciated the ease of participating in the program, compared to

the traditional downstream rebate program, because the discount is applied instantly. This is especially

helpful for smaller projects where the time spent completing and submitting a rebate application might

outweigh the amount of the rebate. One contractor said, “we’ve used the rebate application program

quite a few times, but it’s a pain. The [Midstream Lighting] program is so much easier, especially for a

contractor because everything is done by the distributor.”

7.5.4 Market Effects

Using interview data, Cadmus conducted a qualitative assessment of the Midstream Lighting program’s

effect on sales of efficient products outside of the program.

In PY10, distributors said their stocking practices were driven by customer demand or, in the case of

multistate distributors, corporate policies. Additionally, some distributors partner with manufacturers

and tend to stock products primarily from that preferred manufacturer. Nevertheless, they attributed

some of the increase in customer demand to utility program incentives, including those offered by

Midstream Lighting. This led some distributors to increase the number of program-qualifying products

they stock to keep up with customer demand. Three distributors with stores outside of PPL Electric

Utilities’ territory said that Midstream Lighting had some impact on the products these stores stock. One

said that the “philosophy of [Midstream Lighting] spread to our non-PPL [Electric Utilities’] stores.”

Likewise, most contractors interviewed tended to recommend efficient lighting to all of their clients, but

they also credited Midstream Lighting and distributors’ recommendations in their clients’ decisions to

upgrade their lighting. This finding is consistent with PY9, where 14 of 15 contractors said they always

recommend efficient lighting to their clients and the majority thought that program incentives and

distributors’ recommendations impacted their clients’ decisions.

These findings suggest that, although Midstream Lighting does help drive sales of efficient lighting, its

effects are mostly direct program effects. Even so, in a market that is moving toward energy efficiency,

increasing the rate at which customers choose to upgrade their lighting likely helps to maintain this


momentum. Because distributors influence contractors’ recommendations, their practices can impact

the market outside of the program.

For more detail regarding these specific findings, see Appendix E.2.1 Additional Findings.


Because the Midstream Lighting Program is part of the Non-Residential Energy Efficiency Program, cost-

effectiveness is presented in 5.4 Cost-Effectiveness Reporting.


7.7 Recommendations

Overall, the Midstream Lighting program has shown continued growth in Phase III and had high realization rates. Participating distributors

appreciated the improvements to the reporting process introduced via the new portal-based processing system and few participants reported

issues with the new account number requirement. Recommendations are provided in Table 7-11, along with a summary of how PPL Electric

Utilities plans to address the recommendations.

Conclusion 1: Midstream Lighting participation might be

augmented by more direct or cross-program marketing to

commercial customers.

Findings Support

• Awareness of the Midstream Lighting is still fairly low among participants of other Non-

Residential programs (39%) and the small-commercial general population (25%) survey

respondents. (See section 7.5.3.2 Program Awareness.)

Table 7-11. Status of Recommendations for Midstream Lighting

Midstream Lighting Program

Conclusion Recommendation

EDC Status of Recommendation

(Implemented, Being Considered, Rejected

and Explanation of Action Taken by EDC)

Conclusion 1: Midstream Lighting participation might be

augmented by more direct or cross-program marketing to

commercial customers.

If increased participation is desired, consider working

with the ICSP to market Midstream Lighting outside of

the distributor channel.

Being considered.

The program offers financial incentives to customers who install equipment that is not offered in PPL Electric Utilities’ other commercial programs.

CUSTOM PROGRAM



PY10 saved

164,504 MWh/yr

63,938 MWh/yr





100% satisfied with the professionalismof the program representatives

satisfied with the ease of the online application process82%

satisfied with the information providedabout the application process93%

satisfied with the time it took to process the application

72% satisfied with the ability to trackyour rebates

71%

A total of 72 participants:

PY10 PARTICIPATION

2 Agricultural

Combined Heating and Power (CHP)2

HVAC18

7 Lighting

21 Motors

4 Refrigeration 15 Other unspecified

3 Photovoltaic

Chapter 8 Non-Residential Custom Program PPL Electric Utilities | 80

8 Non-Residential Custom Program

The Custom Program, a component of the Non-Residential Energy Efficiency Program, offers financial

incentives to customers who install equipment that is not offered in PPL Electric Utilities’ other

commercial programs or is not addressed in the PA TRM). Equipment may include new or replacement

energy-efficient products, retrocommissioning, repairs, equipment optimization, new construction,

operational and process improvements, combined heat and power (CHP), and behavioral changes that

result in cost-effective energy savings.

The Custom Program offers incentives for the avoided or reduced energy consumption kWh/yr that

result from the completed project. Incentives are subject to an annual cap for each project ($500,000)

and for each participating customer ($500,000 per customer site per year or $1,000,000 per parent

company per year). Incentives cannot exceed 50% of the total project cost, excluding internal labor

costs.

To qualify, C&I customers are required to submit documentation that their proposed efficiency upgrades

pass the program’s cost-effectiveness threshold, and the project must be approved before equipment is

purchased. Projects with TRC test score of greater than 0.7 are eligible for an incentive.

PPL Electric Utilities pays the incentive to the customer following successful implementation of a cost-

effective project, and the incentive may vary by the type or size of the equipment, system, or

improvement. For projects where expected savings are greater than 500,000 kWh/yr, PPL Electric

Utilities bases the incentive payment on verified savings, rather than reported savings. This approach is

called real-time evaluation and is a cornerstone of the Custom Program.

The ICSP, CLEAResult, manages the program and handles application intake, assesses eligibility, and

calculates project energy savings and incentives.



A PY10 participant is defined as a project that was commercially operable and received an incentive

payment between June 1, 2018, and May 31, 2019.34 Projects for which customers submitted an

application during this period that did not receive an incentive or projects commissioned during this

period that did not receive an incentive are not counted as participants in PY10. Additionally, an

individual customer may have multiple participating projects.

34 As defined by the Phase III Evaluation Framework, EDC-claimed savings are determined by the date the equipment is “installed and commercially operable.” Equipment that is installed and not commissioned, or operating as intended, is not considered “commercially operable.”




payments for the Custom Program in PY10 by customer segment.

Table 8-1. PY10 Custom Program Participation and Reported Impacts

Parameter GNE Large C&I Residential Small C&I Total⁽¹⁾

PYTD # Participants 16 28 1 29 74 (2)

PYRTD MWh/yr 30,328 27,362 17 9,452 67,159

PYRTD MW/yr 2.57 3.29 0.01 1.29 7.16

PYVTD MWh/yr 29,992 26,230 11 7,705 63,938

PYVTD MW/yr 4.66 3.23 0.01 1.19 9.09

PY10 Incentives ($1,000) $1,318 $1,699 $1 $842 $3,860

⁽¹⁾Total may not match the sum of columns due to rounding. (2) PPL Electric Utilities' tracking database includes 74 unique projects in PY10 but two of these are incentive adjustments for projects reported in a previous program year.

Table 8-2 lists the types of projects completed in PY10 and percentage of reported savings.

Table 8-2. PY10 Program Custom Project Types

Project Type Number of Participants

(n=72) (4)

Percentage of Reported Savings Represented by Project Type

(n=100%) (1)

Combined Heating and Power (CHP) (2) 2 27.0%

HVAC 18 16.8%

Lighting 7 15.7%

Other (3) 15 15.4%

Motors 21 13.4%

Photovoltaic 3 7.6%

Refrigeration 4 3.6%

Agricultural 2 0.4% (1) The sum of the column may not add to 100% due to rounding (2) There are two participants for the CHP equipment type in PY10. One of the participants was an initial incentive payment for a project that was operating but not reported in PY10. The other participant represents all the reported savings. (3) The other project type includes C&I improvements such as replacing old mold injection machines, converting from electric to natural gas, process improvements, controls, extruder upgrade, new construction projects, and fan motor improvement. (4) PPL Electric Utilities' tracking database includes 74 unique projects in PY10 but two of these are incentive adjustments for projects reported in a previous program year. The incentive adjustments are not included in the total population in this table.



Table 8-3 shows the evaluation sampling strategy. The target confidence and precision levels for each

stratum were chosen to meet an overall program target of 85% confidence and 15% precision (85/15).

More details are in Appendix F.1 Gross Impact Evaluation.


Table 8-3. PY10 Custom Program Gross Impact Evaluation Sample Design

Stratum Population

Size


Target Sample

Size

Achieved Sample

Size Impact Evaluation Activity

Large 26 (1) Census 26 26

File review, site-specific measurement and verification plans, baseline and post-installation visits, metering installed (if required), verified savings analysis and report

Small 44

CP= 85/20

Cv = 0.64 (assumed)

10 10

File review, site-specific measurement and verification plans, post-installation visit, metering installed (if required), verified savings analysis and report

CHP 2 [2] Census 1 1

File review, site-specific measurement and verification plans, baseline and post-installation visits, metering installed (if required), verified savings analysis and report

Total Participants

72 [3] N/A 37 37

[1] The large stratum population total is 28 including two incentive adjustments. These projects with incentive adjustments without claimed savings that were not included in the sample population for PY10. [2] The CHP stratum includes two participants; savings were claimed for one participant (a project) in PY10. The other participant involved an initial payment of a project that did not claim savings but was operational in PY10. Savings for this project will be claimed in PY11. [3] PPL Electric Utilities' tracking database includes 74 unique projects in PY10 but two of these are incentive adjustments for projects reported in a previous program year. The incentive adjustments are not included in the total population in this table.



Table 8-4. Custom Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 46,368 54,199 (1) 63,938 164,504 (2)

(1) PY9 verified includes savings reported in PY8 but verified in PY9. (2) Phase III verified savings may not match sum of program years due to rounding.

In PY10, the Custom Program reported energy savings of 67,159 MWh/yr, as shown in Table 8-5, and

demand reduction of 7.16 MW/yr, as shown in Table 8-6.


The achieved precision for the program-level results was in compliance with the Evaluation Framework,

exceeding the requirements to meet 85/15 target levels.35 Results for the program overall, including

large, CHP, and small strata, are reported with 2.11% precision at the 85% confidence level.

Table 8-5. PY10 Custom Program Gross Impact Results for Energy

Stratum PYRTD MWh/yr Energy

Realization Rate Sample Cv or Error Ratio

Relative Precision at 85% C.L.

PYVTD MWh/yr

Custom – CHP 18,106 100% 0.00 0.00% 18,106

Custom – Large 39,902 100% 0.00 0.00% 39,902

Custom – Small 9,151 65% 0.55 24.34% 5,930

Program Total⁽¹⁾ 67,159 95% N/A 2.11% 63,938

⁽¹⁾Total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Table 8-6. PY10 Custom Program Gross Impact Results for Demand

Stratum PYRTD MW/yr Demand



PYVTD MW/yr

Custom – CHP 0.00 N/A 0.00 0.00% 2.16

Custom – Large 6.09 100% 0.00 0.00% 6.09

Custom – Small 1.08 78% 0.75 33.30% 0.84

Program Total⁽¹⁾ 7.16 127% N/A 2.86% 9.09

⁽¹⁾Total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

The energy (kWh) realization rate is 100% for the real-time evaluated projects in the large stratum

because savings were verified before the incentive was paid.

For all projects verified in PY10 site visits, Cadmus updated the assumed equipment parameters or

operating sequences used to determine the reported savings and calculated the verified savings (see

Appendix F.1 Gross Impact Evaluation).

For the CHP stratum, the demand savings realization rates were not 100% because of the missing

reported demand savings for the single CHP project claimed in PY10. Verification documentation

submitted to the ICSP stated the correct demand savings.

For the small stratum, several factors led to differences between the reported and verified savings and

to the observed realization rates. Each may have caused an increase or decrease in project energy

savings, depending on the specific circumstances of that project. Further discussion on the sources of

the factors affecting the realization rate is found in Appendix F.1.3 Realization Rate Findings.




The methods used to determine net savings for downstream, upstream, and midstream programs are

provided in the Evaluation Framework,36 which discusses the common methods to determine free

ridership and spillover. Cadmus used self-report surveys, administered online and by phone, to assess

free ridership and spillover for the Custom Program.



Table 8-7 lists the methods and sampling strategy used to determine net savings for the Custom

Program in PY10. Cadmus conducted online and telephone self-report surveys with 21 of 63 Custom

Program participants between February 2019 and July 2019. Additional details about the methodology

are in Appendix Q Net Impact Evaluation and Appendix F.2 Net Impact Evaluation.

Table 8-7. PY10 Custom Program Net Impact Evaluation Sample Design

Stratum Stratum

Boundaries Population(1)


Response Rate (2)

NTG Activity

Custom Participants 74 21 0.28 Self-Report Surveys

Program Total N/A 21 N/A (1) The total population was 74. After selecting unique participants; removing projects without PY10 savings (two incentive adjustments); and removing customers who had participated in a survey in the last three months, did not have valid contact information (email or telephone number), were on the do not call list, or opted out of the online survey, the sample frame was 38. (2) Response rate is calculated as the percentage of respondents who answered the free ridership questions (n=21) divided by the number of records in the population.

Table 8-8 shows the free ridership, spillover, and NTG ratio for the Custom Program. Free ridership for

the Custom Program was 35% in PY10, weighted by the size of the project completed by respondents.

The respondents represented 27% of the program’s verified population savings. Eight respondents had

large stratum projects, 13 had small stratum projects. The CHP participants did not complete a survey,

so Cadmus did not conduct a free ridership analysis. Additional details are in Appendix F.2 Net Impact

Evaluation.



Table 8-8. PY10 Custom Program Net Impact Evaluation Results

Stratum Number of

Surveys Free Ridership

(%)(1) Spillover

(%) NTG Ratio

Relative Precision

at 85% C.L.

Custom (all projects) 21 35% 0% 0.65 15%

Program Total 21 35% 0% 0.65 15%

⁽¹⁾Weighted by verified kWh/yr savings.

Table 8-9 shows PY10 Custom Program free ridership by stratum. The weighted average free ridership

for small stratum projects is high at 62% but the surveyed projects only represent 11% of the analysis

sample verified savings. The overall program free ridership estimate of 35% is heavily weighted towards

the large stratum free ridership of 31% as large stratum respondents represent 89% of the overall

custom analysis sample verified savings.

Table 8-9. PY10 Custom Program Free Ridership Comparison by Stratum

Stratum Number of

Respondents

Weighted Free Ridership

(%)1

Percentage of Analysis Sample Verified Savings

Percentage of Program

Population Stratum Verified

Savings


Custom – Small 13 62% 11% 32% 41%

Custom – Large 8 31% 89% 39% 18%

Program Total 21 35% 36% 15%

⁽¹⁾Weighted by verified kWh/yr savings.

Because custom projects are unique and nearly all are high impact, a separate group of high-impact

projects was not selected for the net savings analysis in PY10. Cadmus did not identify any high-interest

projects that were not already selected into the large, small, or CHP stratums.


Table 8-10 shows the realization rates Cadmus applied to the reported energy and demand savings

estimates to calculate the verified savings estimates for the PY10 Custom Program component of the

Non-Residential Energy Efficiency Program.

Table 8-10. PYTD and P3TD Custom Program Savings Summary

Savings Type Energy

(MWh/yr) ⁽¹⁾ Demand

(MW/yr )⁽¹⁾

PYRTD 67,159 7.16


PYVTD Net (2) 41,560 5.91

P3RTD 166,713 18.70

P3VTD Gross 164,504 20.74

P3VTD Net(2) 119,218 14.92 (1) Total may not match sum of rows due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.




The main research objectives for the PY10 process evaluation of the Custom Program focused on

customer experience and satisfaction, program performance, and program influence.


The PY10 process evaluation activities for the Custom Program included these:


• Online and telephone participant surveys

Table 8-11 lists the process evaluation sampling strategy. Additional details about sampling

methodology are included in Appendix F.3.2 Survey Approach and Appendix R Survey Methodology.

Table 8-11. PY10 Custom Program Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size

Achieved Sample

Size

Number of Records Selected

for Sample Frame (1)




Staff Telephone In-depth Interview

2 N/A (3) 2 2 2 100%

Participants Custom Online

72 (4) N/A (3) All

participants

2 38

100%

Telephone 19 100%

Program Total

74 23 40 100%

(1) Sample frame is a list of participants with contact information who have a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities tracking database for projects that generated savings. After selecting all unique records, Cadmus removed any records from the population if the customers had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (email or telephone number), were on the do not call list, opted out of the online survey, or did not have PY10 savings (incentive adjustments). This left 38 records available to contact for the survey. (2) Percent contacted means the percentage of the sample frame contacted to complete surveys. (3) Because this program’s evaluation did not include sampling, Cv and target precision are not meaningful. (4) PPL Electric Utilities’ tracking database includes 74 unique projects but two of these are for incentive adjustments. The incentive adjustments are not included in the total population in this table.

8.5.3 Survey Methodology

Cadmus conducted online and telephone self-report surveys with 21 of 38 Custom Program participants

between February 2019 and July 2019. To increase response rate, PPL Electric Utilities key account

managers provided additional outreach to non-responders. The online and telephone surveys asked

identical questions to assess satisfaction, net savings, and the influence of the contractor or design

engineer on project design. Additional information is found in Appendix R Survey Methodology.


8.5.3.1 Program Staff and ICSP Interview Methodology

In March 2019, Cadmus conducted interviews with program managers from PPL Electric Utilities and the

ICSP. The interviews focused on identifying and assessing changes to program design and delivery from

PY9 to PY10 and understanding the areas that are working well and any challenges.



The Custom Program was delivered effectively in PY10 and maintains high levels of customer

satisfaction. The ICSP delivered the program in PY10 similar to PY9 except for one modification. PPL

Electric Utilities initially accepted applications for air compressor projects under 75 horsepower (hp)

under the Direct Discount Program in PY9; however, these projects were moved to the Custom Program

in PY10.



Satisfaction among Custom Program respondents was high. Twenty survey respondents (96%; n=21)

were very satisfied or somewhat satisfied with the program in PY10; results were consistent with PY9

(Figure 8-1).

Figure 8-1. PY10 Overall Custom Program Satisfaction

Source: Survey question, “Now, thinking about your overall experience with PPL Electric Utilities' business

Energy efficiency program, how would you rate your satisfaction?”

Participants were most satisfied with the information provided about the application process (85% were

very satisfied; n=13).


Overall, most respondents were satisfied with the time it took to process the application (71% were

satisfied; n=21). However, only 33% of these respondents were very satisfied, the lowest rating for any

program component. Figure 8-2 presents these results.

A number of respondents said that several program components were not applicable to them: 10 said

the application process was not applicable, and 14 said the ability to track rebates was not applicable to

them. This indicates that a third party handled the rebate processes for some respondents.

Figure 8-2. PY10 Participant Satisfaction with Custom Program Components

Source: Survey Question “Please indicate how satisfied you are with each one.”

Percentages may not add up to 100% due to rounding. Respondents who select not applicable are excluded from the analysis.

Areas Working Well

Survey respondents (n=21) said the top three components that worked well in the Custom Program

were the rebates they received, communication with PPL Electric Utilities’ staff or account manager, and

communication with the ICSP (Figure 8-3). On the other hand, only 10% thought that the time it took to

receive their rebate in the mail and the application process worked well. These findings are not

surprising as the ease of the application process and ability to track rebates received the fewest high

satisfaction ratings. These findings are consistent with PY9 survey results37.

37 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to PA PUC. Prepared by

Cadmus. November 15, 2018.


Figure 8-3. PY10 Custom Program Areas that Are Working Well

Source: Question, "Thinking about what worked well with the business energy-efficiency program,


Reasons survey participants completed the project were consistent with PY9 results (Figure 8-4).

Figure 8-4. PY10 Reasons Participants Completed Custom Program Energy Efficiency Project

Source: Survey question “Please describe why your organization completed this project.”

(n=21) Multiple responses allowed.

Figure 8-5 shows the average level of influence different items had on respondents’ decision to

complete the project the way they did, where 5 was extremely influential and 1 was no influence. While

the rating for the influence of PPL Electric Utilities rebates (2.6) decreased from PY9, where the average

influence score was 3.2 (n=25), the difference was not significant.


Figure 8-5. PY10 Average Influence Score for Different Items in the Custom Program

Source: Survey question “Please rate each item on how much influence it had on the decision to complete

the project the way it was completed on a scale from 1 to 5 where 5 is extremely influential and 1 is no influence.”

Additionally, 43% of respondents said the energy or operating costs were the most important criteria in

deciding to move forward and complete the project (n=21), as shown in Figure 8-6.

Figure 8-6. PY10 Most Important Criteria for Completing a Custom Program Project

Source: Survey question “Which of the following criteria was the most important

in deciding whether the project would go forward? (n=21)


Areas for Improvement

Thirteen of 21 respondents provided recommendations to improve the program. The three most

common suggestions for improvement included improving communication about the program and

program requirements, decreasing the rebate processing time, and streamlining the rebate process.

Respondents suggestions included these:

• “More clarity about the information needed to fill out the [application] form; there wasn't a

clear expectation of what [PPL Electric Utilities] needed.”

• “The close out was not smooth it took two to three months for them to chase down the

paperwork for the rebate and the reporting; never got final report.”

• “Somewhat challenging to put together all the needed energy saving aspects of a net zero

energy building hard to characterize savings associated with it.”


Because the Custom component is part of the Non-Residential Energy Efficiency Program, cost

effectiveness is presented in section 5.4 Cost-Effectiveness Reporting.


8.7 Recommendations

Overall, the Custom Program has been highly successful, with the verified savings of 63,938 MWh/year. Nearly all program respondents

(20 of 21) were satisfied with the program.

Recommendations are provided in

Table 8-12, along with a summary of how PPL Electric Utilities plans to address the recommendations.

Conclusion 1: Improving program communication

may help maintain the program’s high satisfaction

ratings into PY11.

Findings Support

• The majority of survey respondents (96%; 20 of 21) were very or somewhat satisfied with the

program overall. The three most common suggestions for improvement included improving

communication about the program and program requirements, decreasing the rebate processing

time, and streamlining the rebate process (n=13). (See section 8.5.4.2 Program Satisfaction.)

Table 8-12. Status of Recommendations for the Custom Program

Custom Program


EDC Status of Recommendation (Implemented, Being

Considered, Rejected and Explanation of Action Taken

by EDC)

Conclusion 1: Improving program communication may

help maintain the program’s high satisfaction ratings

into PY11.

Consider displaying already developed tools and tips

(e.g., FAQ page, CLEAResult contact information, and

other tools) more prominently on the home screen of

the Non-Residential Program website. This may help

improve communication by making these essential

tools more readily visible and available to customers.

Implemented.

CONTINUOUS ENERGY IMPROVEMENT PROGRAMThis program targets school districts, providing technical support for schools to develop an energyefficiency master plan and implement it over two years.


PY10 saved

1,190 MWh/yr

549 MWh/yr


1%of Non-Resident al program savings

of Non-Resident al program savings

1% Satisfied with overall program75%


Areas working well:

incorporation of CEI into broader sustainability efforts

100%

75%

student engagement

PY10 PARTICIPATION

4 school districts1 pilot school in each district

18 participating schools

10 lementary schools4 iddle schools

4 igh schools

Chapter 9 Non-Residential Continuous Energy Improvement Program PPL Electric Utilities | 94

9 Non-Residential Continuous Energy Improvement Program

The Continuous Energy Improvement (CEI) component (hereafter the CEI Program) of the

Non-Residential Energy Efficiency Program targets school districts. PPL Electric Utilities provides

technical support for schools to develop and implement an energy master plan over at least two years.

At the end of each year, school districts receive an incentive of $.06/kWh of savings achieved. Cadmus

verifies savings by analyzing electricity usage.

In PY8, CLEAResult, the ICSP, worked with PPL Electric Utilities to recruit four school districts to

participate during PY9 and PY10. The ICSP helped each district select one pilot school to focus on during

the first year (PY9). The ICSP conducted an audit at the pilot school and developed an energy master

plan along with a methodology for measuring energy savings. The energy master plan included

improvements in equipment and O&M and changes in the energy-related behaviors of staff, faculty, and

students. Each district developed an energy reduction goal. Because the GNE sector was wait-listed for

rebates through the Efficient Equipment and Custom Incentive components of PPL Electric Utilities’ Non-

Residential Program in January, 2018, only equipment upgrades purchased through the Midstream

Lighting component were eligible for incentives in PY10.

In the first year (PY9), each district identified an energy manager, who could be a facility manager,

energy expert, teacher, or administrator. These energy managers participated in the PY9 pilot school

audits with the ICSP, in preparation for conducting their own audits at other schools in the district in the

second year. In PY10, with support from the ICSP, the district energy managers worked with the other

schools in their districts to implement the CEI activities. The district energy managers continued to

collaborate and share best practices with each other during monthly conference calls led by the ICSP.



A participant in the CEI Program is defined as an individual school and is listed in PPL Electric Utilities’

tracking database as an individual account number.38 The name of the school district is also recorded in

the tracking database.

In PY10, the same four school districts that participated in PY9 continued their participation. The four

districts have a total of 18 participating schools that reported savings in PY10 (10 elementary schools,

four middle schools, and four high schools). In PY9, each district chose one pilot school—two were

elementary schools and two were high schools. However, no PY9 savings were reported for one of the

pilot elementary schools and those savings were reported in PY10.

The types of schools in each district are listed in Table 9-1.

38 Some schools have multiple PPL Electric Utilities’ accounts, but only one account number for each participating school is shown in the tracking data.


Table 9-1. Participating Schools in Each District in the Continuous Energy Improvement Program

District Elementary

Schools Middle Schools

High Schools

Total

1 3 1 1 5

2 3 1 1 5

3 4 1 1 6

4 0 1 1 2

Total 10 4 4 18(1)

⁽¹⁾ For reporting purposes, a participant is defined as a unique job, and one school in PY10 was associated with multiple jobs, because it reported negative savings from PY9 in PY10 Q2. Therefore, the total in this table does not match the total reported participants in Table 9-2.


Table 9-2 presents the participation counts and reported energy and demand savings by customer

segment for the CEI Program in PY10. Because the program targets school districts, all participants are in

the GNE sector.

Table 9-2. PY10 Continuous Energy Improvement Program Participation and Reported Impacts

Parameter GNE Total ⁽¹⁾

PYTD # Participants (2) 19 19

PYRTD MWh/yr 144 144

PYRTD MW/yr 0.00 0.00

PYVTD MWh/yr 549 549

PYVTD MW/yr 0.17 0.17

⁽¹⁾Total may not match sum of columns due to rounding. (2) In this table, a participant is defined as a unique job. The total does not match the total verified participants in Table 9-1 because one school participated in PY9 but reported savings in PY10.



Cadmus used statistical modeling to estimate energy and demand savings for the program. Details are in

G.1 Gross Impact Evaluation.



Table 9-3. Continuous Energy Improvement Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr - 641 549 1,190 (1)



In PY10, the CEI Program reported energy savings of 144 MWh/yr, as shown in Table 9-4, and demand

reduction of 0 MW/yr, as shown in Table 9-5. Cadmus included all participating schools in its savings

verification analysis.

Table 9-4. Continuous Energy Improvement Program Gross Impact Results for Energy


Realization Rate Sample Cv or

Error Ratio


85% C.L.

PYVTD MWh/yr (1)


144 382% N/A 11.91% 549

Program Total (2) 144 382% N/A 11.91% 549 (1)Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings ⁽2⁾Total may not match sum of rows due to rounding.

Table 9-5. Continuous Energy Improvement Program Gross Impact Results for Demand




85% C.L.

PYVTD MW/yr (1)


0.00 N/A 0.17 8.59% 0.17

Program Total⁽2⁾ 0.00 N/A 0.17 8.59% 0.17 (1)Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings ⁽2⁾Total may not match sum of rows due to rounding.

The energy savings realization rate was 382%. Many factors contributed to the variations between

ex ante and ex post savings, including the frequency of data used, modeling techniques, selection of

independent variables, calculation of heating degree days (HDDs) and cooling degree days (CDDs), and

capital project savings. The ICSP’s reported savings were calculated using monthly billing data, and all

modeled independent variables were also sampled monthly, whereas Cadmus used hourly AMI and

weather data which was aggregated at daily intervals.

The higher frequency data Cadmus used allowed for a greater selection of model inputs and yielded

greater explanatory power of the resulting models. This choice of datasets also facilitates the use of

advanced machine learning regression techniques that would otherwise offer marginal improvements

when applied to monthly data. However, the largest contributing factor to the high realization rate in

PY10 is that the ICSP subtracted PY9 reported savings from their modeled estimate for one school.

Cadmus, in accordance with the evaluation plan, considers the measure life for the PY9 savings to be

one year, thus allowing for cumulative savings to be claimed each year.

Cadmus did not calculate a realization rate for demand savings because the ICSP did not report demand

savings.


Cadmus assesses net savings only to inform future program planning. Energy savings and demand

reduction compliance targets are met using verified gross savings. Cadmus followed the Evaluation


Framework’s recommended method for self-report surveys.39 Additional information about the net

savings evaluation is in Appendix Q Net Impact Evaluation.

In accordance with the evaluation plan, Cadmus assessed net savings qualitatively in PY10. Cadmus

interviewed energy managers at all four districts between April and May 2019 (Table 9-6). The net

savings questions focused on program activities conducted within each district during PY10.

Because a billing analysis was used to estimate energy savings, Cadmus included savings attributable to

free ridership and spillover in the savings estimates for the schools in the four districts.

Table 9-6. Continuous Energy Improvement Net Impact Evaluation Sample Design

Stratum Stratum


Size Achieved

Sample Size Response

Rate (1) NTG Activity

School Districts School districts 4 4 100% Phone Interview

Program Total Total 4 4 100% Phone Interview

(1) Response rate is calculated as the percentage of respondents who completed the interview (n=4) divided by the number of unique records in the population (n=4).


Table 9-7 shows the verified savings for the PY10 CEI Program.

Table 9-7. PYTD and P3TD Continuous Energy Improvement Program Savings Summary

Savings Type Energy

(MWh/yr)⁽¹⁾ Total Demand

(MW/yr)⁽¹⁾

PYRTD 144 0.00

PYVTD Gross 549 0.17

PYVTD Net⁽²⁾ 549 0.17

P3RTD 867 0.00

P3VTD Gross 1,190 0.60

P3VTD Net⁽²⁾ 1,190 0.60

⁽¹⁾ Total may not match sum of rows due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.





The main research objectives for the Continuous Energy Improvement component of the Non-Residential Energy Efficiency Program focused on these questions:

• How satisfied are energy managers with

their CEI program experience?

• What more could the program do to help

energy managers reach their CEI goals?

• In what areas is the program succeeding?

• How closely is program delivery adhering

to program design?

• What challenges are energy managers

facing in implementing behavior change?


The PY10 process evaluation for the Continuous Energy Improvement component included these

activities:

• Interviews with PPL Electric Utilities and

ICSP program managers

• Telephone interviews with the energy

manager representing each of the four

participating school districts

The research activities were consistent with the evaluation plan with one exception:

• Because the program documentation and materials did not change from PY9, Cadmus did not

conduct a document review again in PY10. The interviews with program staff provided enough

information to update the PY10 participant interview guide.

Table 9-8 lists the process evaluation sampling strategy. Each of the four participating school districts

has one energy manager leading the Continuous Energy Improvement activities. The energy manager is

the primary contact for all process evaluation activities.

Table 9-8. PY10 Continuous Energy Improvement Component Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion or Cv in Sample

Design

Target Sample

Size

Achieved Sample

Size

Records Selected



Achieve Sample (2)


Staff Phone

interview 2 N/A (3) 2 2 2 100%

School Districts

School district energy

managers

Phone interview

4 N/A (3) 4 4 4 100%

Program Total

6 6 6 6 100%

(1) Sample frame is a list of participants with contact information who have a chance to complete the interview. (2) Percent contacted means the percentage of the sample frame contacted to complete interviews. (3) Because this program’s evaluation did not include sampling, Cv and target precision are not meaningful.



To launch the PY10 evaluation, Cadmus conducted interviews with the program managers from PPL

Electric Utilities and the ICSP in November and December 2018. The interviews discussed operations and

identified areas that are working well and areas that may benefit from change.

9.5.2.2 Participant Interview Methodology

In April and May of 2019, Cadmus conducted telephone interviews with the energy manager of each of

the four participating school districts. These interviews focused on the topics described above in section

9.5.1 Research Objectives.


In this section, Cadmus presents interview findings.


PPL Electric Utilities, the ICSP, and the energy managers reported that CEI is operating well but would

have benefited from additional support in engaging other schools in their districts to implement

behavioral initiatives.

In PY10, three of the four energy managers conducted audits at schools other than the pilot school. One

energy manager reported not being able to implement CEI activities in additional schools, primarily

because of time constraints, but noted involvement in the program strongly influenced the energy

efficiency design considerations being implemented in the construction of a new school in the district.

Other Program Participation

Cadmus determined whether the schools in the CEI component participated in other PPL Electric

Utilities’ programs during PY10. Savings from projects completed under other programs must be

subtracted from the school’s CEI savings to avoid double-counting. Also, participation in other programs

indicates that the energy managers are aware of PPL Electric Utilities’ other offerings.

During PY10, three schools in two districts purchased LED linear lamps through the Midstream Lighting

component.

During PY10, as in PY9, only one school also participated in PPL Electric Utilities’ Student Energy Efficient

Education (SEEE) Program. The energy manager for this district said there had been no collaboration

between representatives running the two programs. PPL Electric Utilities noted that, although Cadmus

had recommended in PY9 that PPL Electric Utilities consider cross-promoting these programs to

encourage collaboration, this would require connecting teachers who participate in the SEEE Program

(delivered through another channel) with the energy manager in their district. Due to the timing of

Cadmus’ evaluation and the planned sunset of the CEI component after PY10, it was not practical to

implement this recommendation.


9.5.3.2 Overall Satisfaction

Satisfaction with the program overall declined in PY10. All four energy managers said they were very

satisfied with the program in PY9, but in PY10 only one gave this highest satisfaction rating. Two rated

their satisfaction as 4 out of 5, or somewhat satisfied, and one would have ranked satisfaction as a 4 but

dropped it to a 3 (neither satisfied nor dissatisfied) by the end of the year because contact with the

program was less frequent. All four said they were very satisfied with their experience with the ICSP.

Nevertheless, two who indicated they were very satisfied when the advisor was present said they would

have benefited from more frequent visits or interactions. These two rated their satisfaction with the

level of support they received from their advisor as a 3 (neither satisfied nor dissatisfied).

Areas Working Well

PPL Electric Utilities noted that many individual participating schools were achieving energy savings. The

ICSP and energy managers agreed they had a good relationship and communicated well. Three energy

managers said schools had incorporated CEI activities into broader sustainability efforts that included

non-energy components such as recycling or food and water waste reduction. One energy manager

publicly presented money from PY9 program incentives to schools that had achieved savings as a way to

encourage additional schools to participate.

As in PY9, energy managers reported that students, once engaged, were enthusiastic about and

motivated by the CEI behavioral initiatives. At one high school, a club was formed as part of the program

in PY9, and it met more regularly in PY10. Another said that, although behavioral change continues to be

a challenge, the continued incorporation of initiatives into educational curriculum helps to increase

involvement.

These are some examples of activities that schools have implemented:

• Incorporation of energy education into existing curriculum

• Incorporation of behavioral initiatives into broader, ongoing sustainability initiatives

• Student-led lighting audits

• In-school surveys and quizzes about energy usage administered and filmed by the high school

television studio

• Middle school special education student-led audit of classroom equipment left on

• Middle school poster and video contests

Challenges

PPL Electric Utilities, the ICSP, and the energy managers reported lack of time, support, and human

resources as the primary challenges in expanding the program beyond the pilot school in PY10. As in

PY9, energy managers stressed that continual engagement with the ICSP was important in maintaining

focus on the program, but in PY10 energy managers reported that the frequency of contact, especially

in-person contact, had dropped off over the year.


Lack of Resources

Two of the four energy managers specifically cited their own lack of time as a challenge in implementing

CEI activities. A third said that workload had increased substantially, especially in PY10, as a result of

participating in the program. Three of the four said they could have used additional support from the

ICSP in PY10.

Maintaining Engagement with Students and Staff

In PY9 and PY10, energy managers said changing culture and/or habits was both an objective and a

challenge in implementing behavior-based initiatives, which rely on cooperation among students,

teachers, and facility maintenance staff, all of whom are busy and have competing priorities. One energy

manager said in PY10 that, initially, high school students had the least interest in the program,

compared with students in other grades, but once engaged they ultimately embraced the program.

Lack of Control Over Building Systems

PPL Electric Utilities and energy managers said schools found it difficult to implement operational

controls because they could not change operating hours and activities that impact the school’s

consumption.

Suggestions for Improvement

Energy managers were asked how CEI could help address their challenges. All suggested that more

regular communication and support from the ICSP would help them maintain engagement with the

program. Three referred specifically to in-person, on-site visits to engage directly with students and

teachers. One wished for a more convenient way (other than the conference calls) to communicate

regularly with other groups and suggested establishing a web portal.


Because the Continuous Energy Improvement component is part of the Non-Residential Energy

Efficiency Program, cost-effectiveness is presented in section 5.4 Cost-Effectiveness Reporting.


9.7 Recommendations

Overall, the CEI Program saved energy. PPL Electric Utilities and the ICSP reported it is operating well, and all four energy managers said they

were very satisfied and would recommend it to other school districts. Recommendations are provided in Table 9-9, along with a summary of how

PPL Electric Utilities plans to address the recommendations.

Conclusion 1: Participating districts would benefit

from initial guidance from the ICSP on engaging

and motivating additional administrators and

staff to implement program initiatives. Regular

visits from the ICSP would also help districts to

maintain engagement and interest in the

program.

Findings Support

• Energy managers struggled to find the time to engage internal resources to help expand the

program in PY10, while at the same time experiencing a decline in the frequency of engagement

from the ICSP. (See section 9.5.3.2 Overall Satisfaction.)

Table 9-9. Status of Recommendations for Continuous Energy Improvement Program




Considered, Rejected and Explanation of Action Taken by

EDC)

Conclusion 1: Participating districts would benefit from initial guidance from the ICSP on engaging and motivating additional administrators and staff to implement program initiatives. Regular visits from the ICSP would also help districts to maintain engagement and interest in the program.

In future programming, consider including a training component covering recommended strategies for schools to market the program internally to increase the level of commitment from additional human resources. This is likely to help schools be better prepared and more self-sufficient when it comes time to expand the program beyond its pilot year. Ensure the ICSP continues to conduct regular in-person visits with participating schools to maintain interest in the program.

Being considered.

EFFICIENT LIGHTING PROGRAM

PY10 expenditures

85%

72%

of projected

of projected


Phase III expenditures so far$2 ,561

$5,283


PY10 saved

379,329 MWh/yr

105,364 MWh/yr

130%

164%

of projected

of projected


The program encourages residential customers to purchase and install LED bulbs by providing upstream incentives to participating manufacturers to discount the prices of a variety of bulbs sold at local retail stores.

A total of

bulbs sold

2,584,462

17%

67%

PY10 BULBS SOLD

LED A-line bulbs

LED reflector bulbs

LED candelabra bulbs

LED globes

LED fixtures

from home improvement stores

from membership club stores

from hardware stores

from mass merchandise stores

from discount stores

9%

4%

4%

47%

16%

16%

11%

7%

1% from lighting and electronics stores

from other s ores (including giveaways)2%

BULB TYPE

RETAIL CHANNEL

Chapter 10 Efficient Lighting Program PPL Electric Utilities | 104

10 Efficient Lighting Program

The Efficient Lighting Program encourages residential customers to purchase and install LED bulbs by

buying down the price of program-qualified ENERGY STAR® LEDs. The program provides upstream

incentives to participating manufacturers to discount the prices of a variety of bulbs sold at local retail

stores. The program targets residential customers but is available to all PPL Electric Utilities customers

and anyone who purchases discounted bulbs from participating retailers.

The program is primarily delivered by providing upstream incentives, but includes occasional giveaway

events, where bulbs are given directly to customers at no cost. The ICSP, CLEAResult, managed program

operations and provided support to participating retailers and manufacturers.

The objectives of the Efficient Lighting Program are these:40

• Provide a mechanism for customers to

easily obtain discounted LED bulbs in local

retail stores

• Develop and execute strategies aimed at

transforming the market for LED bulbs

• Obtain approximately 293,000 MWh/year

gross verified savings in Phase III

• Achieve high customer, retailer, and

manufacturer satisfaction with the

program

• Achieve widespread visibility of discounts

through independent and regional retailers

that carry program-eligible LED bulbs

• Engage retailers by educating and training

retail sales associates about LED bulbs

• Educate customers on new lighting

technologies



A participant is a person or business purchasing discounted bulbs. Because of the upstream design of

the Efficient Lighting Program, the identities of purchasers are not known. Cadmus estimated the

number of participants by dividing the total number of bulbs discounted or given away in PY10 by a

bulb-per-participant count derived from residential and commercial customer telephone survey data

collected in PY10.



payments for the Efficient Lighting Program in PY10 by customer segment. The residential sector

accounted for 94% of bulbs sold and 87% of energy savings, while the commercial sector accounted for

40 Program objectives are listed in PPL Electric Utilities revised Energy Efficiency and Conservation Plan Act 129 Phase III. Docket No. M-2015-2515642. November 2018.


6% of sales and 13% of savings. The process for allocating program sales and savings by customer

segment is described in further detail in section 10.2.2.2 Cross-Sector Sales Estimation.

Table 10-1. PY10 Efficient Lighting Program Participation and Reported Impacts

Parameter Residential Small C&I Total (1)

PYTD # Participants (2) 249,116 9,716 258,832

Reported Quantity (bulbs) 2,327,144 257,318 2,584,462

Verified Quantity (bulbs) 2,430,071 154,391 2,584,462

PYRTD MWh/yr 85,374 24,619 109,993

PYRTD MW/yr 9.84 5.08 14.92

PYVTD MWh/yr 91,958 13,405 105,364

PYVTD MW/yr 10.60 2.95 13.55

PY10 Incentives ($1,000) $2,980 $350 $3,330 (1) Total may not match sum of columns due to rounding. (2) The reported participant counts by sector use verified quantities divided by bulbs-per-participation assumptions, as described in Appendix A Upstream Lighting Cross-Sector Sales.



To evaluate the gross impacts of the Efficient Lighting Program, Cadmus audited lighting manufacturer

invoices, verified the ENERGY STAR certification of all bulbs sold through the program, reviewed

program tracking data, and conducted an updated cross-sector sales analysis. Each of these tasks is

discussed in further detail in section 10.2.2.1 Database Review, section 10.2.2.2 Cross-Sector Sales

Estimation, and Appendix H Evaluation Detail – Efficient Lighting Program.


Cadmus collected data through the PPL Electric Utilities’ tracking database, manufacturer invoices

received from the ICSP, and ICSP reports to verify energy savings.


The impact evaluation sampling strategy is summarized in Table 10-2. The impact evaluation activities

produced results with ±1.86% precision at 85% confidence.


Table 10-2. PY10 Efficient Lighting Program Gross Impact Evaluation Sample Design

Stratum Population

Size




Tracking data 269,061(1) N/A N/A Census database review, QA/QC, and ex post adjustments

Manufacturer invoices 688 N/A 70 Manufacturer invoice audit, strategic sample

Residential general population survey

~1.2 million 90/5 300 Cross-sector sales analysis

Small business general population survey

~180,000 90/5 398 Cross-sector sales analysis

(1) Number of records (typically measured as all the information associated with a specific SKU for a given month and retail location, including its quantity sold) in PPL Electric Utilities’ tracking database.


Table 10-3 shows the program’s verified gross program savings.

Table 10-3. PY10 Efficient Lighting Verified Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 145,929 128,036 (1) 105,364 379,329 (2)

⁽1⁾ 86,127 of PSA MWh/yr from Efficient Lighting Program are attributed to small C&I. PY9 verified savings for Efficient Lighting Program were reduced by 282 MWh/yr to conform with the SWE’s PY9 Annual Report findings. The adjustment was divided proportionally (based on PY9 verified savings) between residential (78%) and small C&I sectors (22%). From Pennsylvania Public Utility Commission. SWE Annual Report Act 129 Program Year 9. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, Brightline Group, and Demand Side Analytics, LLC. February 28, 2019. (2) Phase III verified savings may not match sum of program years due to rounding.

In PY10, the Efficient Lighting Program reported energy savings of 109,993 MWh/yr, as shown in Table

10-4, and demand reduction of 14.92 MW/yr, as shown in Table 10-5.

Table 10-4. PY10 Efficient Lighting Program Gross Impact Results for Energy



Relative Precision

at 85% C.L. PYVTD MWh/yr

Upstream Lighting 109,993 96% N/A 1.86% 105,364

Total (1) 109,993 96% N/A 105,364 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Table 10-5. PY10 Efficient Lighting Program Gross Impact Results for Demand



Relative Precision

at 85% C.L. PYVTD MW/yr

Upstream Lighting 14.92 91% N/A 1.86% 13.55

Total 14.92 91% N/A 13.55 (1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.


Baseline wattage adjustments (introduced in 10.2.2.1 ENERGY STAR Verification and discussed in further

detail in Appendix H Evaluation Detail – Efficient Lighting Program) increased program energy savings by

roughly 2,500 MWh/yr (2%). However, cross-sector sales adjustments (described in section 10.2.2.2

Cross-Sector Sales Estimation)—which included changes to assumptions related to commercial sector

hours of use (HOU), coincidence factor (CF), in-service rate (ISR), and proportion of program sales—

decreased program energy savings by more than 7,000 MWh/yr (7%). Overall, these adjustments

produced a net four percent decrease in verified savings compared to reported savings.

10.2.2.1 Database Review

Cadmus reviewed the PPL Electric Utilities’ tracking database, manufacturer invoices, and ICSP reports

to ensure consistency across all data sources, as detailed below.

Lighting Manufacturer Invoice Audit

Cadmus audited copies of lighting manufacturer invoices provided by the ICSP to ensure PPL Electric

Utilities’ tracking database records matched the invoices provided by manufacturers. Cadmus sampled

70 invoices, as shown in Table 10-6, and confirmed the bulb model numbers and quantities from the

tracking database matched the counts reported in the invoices provided by the ICSP. This review found

no errors.

Table 10-6. PY10 Efficient Lighting Program Manufacturer Invoice Sample Sizes

PY10 Quarter(s) Invoice Count Invoices Sampled

Q1 and Q2 344 30

Q3 134 20

Q4 210 20

Total 688 70

For products indicated in PPL Electric Utilities’ database as multi-packs, Cadmus verified the number of

bulbs per pack using the manufacturer invoices. If invoices did not include pack size information,

Cadmus researched bulb information online via the websites of manufacturers, traditional brick-and-

mortar retailers (e.g., The Home Depot), or other online retailers (e.g., Amazon). Cadmus identified nine

stock keeping units (SKUs) listed in PPL Electric Utilities’ tracking database (out of 281 reviewed) that

had different pack sizes than the manufacturer invoices.

However, Cadmus could not verify the pack sizes, likely because the SKUs were discontinued. Therefore,

Cadmus did not update the pack sizes in the tracking database. The nine SKUs comprised only 126 LEDs

sold in PY10, so any changes to pack sizes would have a minimal impact on program savings.

ENERGY STAR Verification

The program strives to offer incentives exclusively for ENERGY STAR lighting products. Using ENERGY

STAR identification numbers or model numbers of every bulb tracked in PPL Electric Utilities’ tracking

database, Cadmus confirmed all bulbs sold through the program met current ENERGY STAR certification

criteria.


Cadmus used the following fields from ENERGY STAR’s Qualified Products List (QPL) as critical impact

evaluation inputs for identifying EISA-exempt bulbs and determining baseline wattages:

• Lumens rating

• Bulb shape (A19, R20, etc.)

• Base type (E26, E12, etc.)

• Diameter (inches)

• 3-way (“yes” or “no”)

Cadmus also used the QPL’s efficient wattage ratings in its calculations to verify savings.

Tracking Data Review

Cadmus reviewed PPL Electric Utilities’ tracking database files and compared them to the QPL to finalize

impact evaluation inputs. Cadmus verified the following:

• Bulb-specific inputs such as bulb type, lumens, and wattages were consistent across all records

for the same SKU.

• Reported wattages were consistent with the wattages provided in bulb type descriptions.

Cadmus also compared PPL Electric Utilities’ tracking database files to ICSP and SWE reports to confirm

consistency and reasonableness of data.

Prior to computing savings using PA TRM algorithms, Cadmus checked baseline wattages for each SKU

against the baseline tables by bulb type in PA TRM Tables 2-2, 2-3, and 2-4 and made adjustments for

records that did not align with these tables.41 For bulbs with lumen ratings outside the ranges specified

in PA TRM Tables 2-2, 2-3, and 2-4, Cadmus used manufacturer-rated baseline wattages, as stipulated in

the PA TRM.

Cadmus adjusted baselines for roughly 11% of program bulbs, mostly to account for specialty bulb types

exempt from the Energy Independence and Security Act (EISA) ruling covering general service lamps

(GSLs), such as 3-way LEDs and candelabra bulbs with medium bases (E26), and for some reflector lamp

types. These adjustments, in aggregate, increased energy and demand savings by 26% among adjusted

bulbs but increased overall program savings by 2%. All baseline adjustments are documented in

Appendix H.1.1 Baseline Adjustments.

10.2.2.2 Cross-Sector Sales Estimation

Cadmus conducted a cross-sector sales analysis in PY10 and applied the updated results to PY10 data.

The analysis produced updated assumptions for commercial sector impact evaluation inputs, including

HOU (6.07), CF (0.409), and proportion of sales (6%). Cadmus applied the default ISR of 98% to the

commercial sector, per instructions in the SWE Cross-Sector Sales Guidance Memo dated May 7, 2019.

41 Pennsylvania Public Utility Commission. 2016 Technical Reference Manual. Act 129 Energy Efficiency and Conservation Program & Act 213 Alternative Energy Portfolio Standards. June 2016. Available online: http://www.puc.pa.gov/filing_resources/issues_laws_regulations/act_129_information/technical_reference_manual.aspx


To learn more about the methodology and results of the PY10 analysis, refer to Appendix A Upstream

Lighting Cross-Sector Sales.


For the program’s PY10 net savings results, Cadmus used the results from its PY8 net-to-gross (NTG)

analysis, which used demand elasticity modeling to estimate participant free ridership of 17% and a net-

to-gross ratio of 0.83. The methodology and results of demand elasticity modeling in PY8 are discussed

in the PY8 Annual Report.42

Cadmus tracked market progress indicators (MPIs), such as changes in sales and purchase patterns, and

satisfaction with and importance of LED traits. When comparing PY8 and PY10 survey results, Cadmus

found several changes in marketplace behavior.

• Awareness of the program among businesses customers increased. Small business customers’

awareness of PPL Electric Utilities lighting discounts offered through participating retailers

increased statistically significantly from PY8 to PY10 (discussed in the Awareness of Program,

LEDs, and Energy Efficiency section).43 However, the percentage of LEDs purchased in retail

stores by small business customers during a promotion or sponsored sale—in which customers

were aware they were participating—decreased from 51% in PY8 to 12% in PY10.

• Shopping habits of business customers have shifted. Small business customers purchased LEDs

online more frequently and at retail stores less frequently in PY10 compared to PY8. These

statistically significant changes are discussed in the Light Bulb Purchase Patterns section (Figure

10-2).44

• Factors important to purchasing decisions have changed slightly. The percentage of residential

customers who consider energy use a very important trait while making the decision to

purchase LEDs significantly decreased from PY8 to PY10, as described in the Importance of LED

Traits to Purchase Decision section (Figure 10-6). Among small business customers, the

importance of longevity, cost, and energy use as factors in LED purchase decisions have

decreased statistically significantly since PY6, but not since PY8 (Figure 10-8).45

Other elements of customer awareness of the Efficient Lighting Program, satisfaction with LEDs and

various lighting traits, and opinion of PPL Electric Utilities have remained consistent from PY8 to PY10.


43 Two-sample t-test for the difference in proportions, statistically significant at 90% confidence.






demand savings estimates to calculate the verified savings estimates for the program in PY10.

Table 10-7. PYTD and P3TD Efficient Lighting Program Savings Summary

Savings Type Energy

(MWh/yr) (1) ),(2) Total Demand

(MW/yr) (1

PYRTD 109,993 14.92


PYVTD Net (3) 87,452 11.24

P3RTD 388,667 55.08

P3VTD Gross (2) 379,329 50.74

P3VTD Net (3) 315,077 42.15 (1) May not match due to rounding. (2) 86,127 of PSA MWh/yr from Efficient Lighting were attributed to Small C&I. PY9 verified savings for Efficient Lighting were reduced by 282 MWh/yr to conform with the SWE’s PY9 Annual Report findings. The adjustment was divided proportionally (based on PY9 verified savings) between residential (78%) and small C&I sectors (22%). (3) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.



The primary purpose of the process evaluation was to administer general population surveys to

residential and small business customers to update the cross-sector sales analysis, update market

progress indicators, and capture inputs for the PY10 impact evaluation.


The PY10 process evaluation activities for the Efficient Lighting Program were these:

• Interview three PPL Electric Utilities and ICSP program staff

• Review the program database

• Conduct a general residential population telephone survey

• Conduct a general small commercial population telephone survey

The research activities in PY10 were consistent with the evaluation plan.

Table 10-8 lists the process evaluation sampling strategy. Additional details about Cadmus’ approach to

contacting customers and the sample attrition are presented in Appendix H.2.1 Survey Approach and

Appendix R Survey Methodology.


Table 10-8. Efficient Lighting Program Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size

Achieved Sample Size (1)

Records Selected

for Sample

Frame (2)


Contacted to Achieve

Sample (3)

Program Staff and ICSP

PPL Electric Utilities and ICSP staff

Phone 3 N/A 3 3 N/A N/A

General Population

Residential Phone ~1.23

million 90/5 300 300 918,909 1%

Small C&I Phone ~180,000 90/5 300 398 94,015 18% (1) Number includes only completed surveys. Respondents could skip questions. (2) Sample frame is a list of active customers with phone contact information drawn from the PPL Electric Utilities’ database. After selecting all unique records, Cadmus removed any records from the population if the customers had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (telephone number), or were on the do not call list. (3) Percent contacted means the percentage of the sample frame contacted to complete surveys.

10.5.2.1 General Population Surveys

Because the Efficient Lighting Program offers upstream incentives to manufacturers, customers who

purchased program-discounted bulbs in stores may have been unaware they purchased bulbs sponsored

by a PPL Electric Utilities program. In May and June 2019, Cadmus conducted surveys with residential

and small business customers to measure market progress indicators such as awareness, knowledge,

and satisfaction with energy-efficient lighting, purchase patterns and usage, and customer

demographics and to calculate cross-sector sales.

The residential survey with a random sample of 300 respondents yielded 90 customers who purchased

LEDs at participating retailers within the last six months. The small commercial survey with a random

sample of 398 respondents yielded 70 small businesses that purchased LEDs at participating retailers in

the last six months. These subsets of purchasers were asked additional questions about bulbs and usage.


Cadmus conducted interviews with PPL Electric Utilities and the ICSP midway through PY10 to monitor

program progress and delivery strategies and to learn of any changes to the program.


The Efficient Lighting Program exceeded its Phase III estimated savings in PY10. Although PPL Electric

Utilities increased incentives for fixtures and other non-general purpose LEDs, it tapered down total

program incentives to slow program sales. In PY10, program sales decreased by 14% (roughly 425,000

units) compared to PY9.

10.5.3.1 Sales by Retail Channel and Bulb Type

Table 10-9 shows program sales by retail channel in PY9 and PY10. In PY10, PPL Electric Utilities sold

nearly half of program bulbs through home improvement stores, up from 26% in PY9 to 47% of total


bulb sales. This represented a shift in sales away from hardware stores (from 31% in PY9 to 16% in

PY10), which PPL Electric Utilities expected to occur. With the program performing well, the ICSP scaled

back additional promotions and incentives provided to hardware stores that were used to stimulate

sales in PY9. The ICSP also attributed stronger sales for home improvement stores to some individual

changes in the stores’ bulb inventory and their own promotions.

The change in distribution of LEDs sold by retailer type from PY9 to PY10 is statistically significant.46 The

program also gave away (at no cost) more than 11,000 bulbs in PY10, comprising less than 0.5% of all

program bulbs.

Table 10-9. Program Unit Sales by Retail Channel and Program Year

Retail Channel PY10 Program Unit Sales PY9 Program Unit Sales(1)

Count Percentage Count Percentage

Home Improvement 1,202,691 47% 784,766 26%

Membership Club 415,088 16% 490,089 16%

Hardware 409,572 16% 928,482 31%

Mass Merchandise 292,690 11% 400,938 13%

Discount 174,066 7% 261,217 9%

Other 43,261 2% 86,023 3%

Lighting & Electronics 35,814 1% 58,278 2%

Giveaway Bulbs 11,280 < 1% 1,584 < 1%

Program Total 2,584,462 100% 3,011,377 100% (1) Retail channel categorization for both PY9 and PY10 were updated to match the categorization used in the SWE Annual Report Act 129 Program Year 9. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, Brightline Group, and Demand Side Analytics, LLC February 28, 2019.

Table 10-10 shows sales by bulb type in PY9 and PY10. The ICSP continued expanding its list of LED

fixtures eligible for incentives, as requested by PPL Electric Utilities. As a result, the program sold more

than 100,000 fixtures in PY10, representing a 460% increase in fixtures sold compared to PY9. Small

increases in the shares of candelabra, fixture, and globe LEDs displaced A-line and reflector LED sales.

Table 10-10. Program Sales by Bulb Type and Program Year

Bulb Type PY10 Program Sales PY9 Program Sales

Count Percentage Count Percentage

LED A-Line (1) 1,724,824 67% 2,144,613 71%

LED Reflector 430,384 17% 560,997 19%

LED Candelabra 223,278 9% 219,182 7%

LED Fixture 108,933 4% 23,418 1%

LED Globe 97,043 4% 63,167 2%

Program Total 2,584,462 100% 3,011,377 100%

(1) A-line LEDs include 14,312 three-way bulbs.

46 Chi-square test, statistically significant at 99% confidence.


10.5.3.2 Participant Profile

Residential General Population

Cadmus used the residential general population survey to calculate the percentage of respondents who

recently purchased LEDs. According to self-reported survey data, 37% of all residential respondents

purchased LEDs (31% from participating retailers and 6% from other retailers). Eighty-four percent of

LED purchasers bought 82% of LEDs from participating retailers. With one exception, these results were

consistent with PY8 results, the last program year during which Cadmus administered general

population surveys. The increase in the percentage of residential respondents who purchased LEDs from

participating retailers was statistically significant (from 24% in PY8 to 31% in PY10).47

Table 10-11 summarizes the breakdown of LED purchase patterns.

Table 10-11. Residential General Population Recent LED Purchasers and Non-Purchasers

Customer Base Respondent Type Percentage of Respondents Percentage of

LEDs Purchased

Residential General Population (n=300)

LED purchasers (within six months of survey)

From participating retailers 31% 82%

From other retailers 6% 18%

LED purchasers (more than six months ago) and non-purchasers 63% N/A

Residential respondents who recently purchased or currently use LEDs (LED users) were more likely to

live in a single-family home (72%) than non-users (57%) and to earn annual household incomes of at

least $50,000 (68%, compared to 45% of non-users). Consistent with PY8 results, housing types and

income levels were different between LED users and non-users by statistically significant margins.48

Ninety-three percent of residential respondents possessed at least a high school degree or equivalent,

and 38% possessed at least a four-year college degree. Differences in education between LED users and

non-users were small and not statistically significant.

Small Business General Population Survey

Prior to initiating the small business general population survey, respondents were asked to indicate who

makes lighting purchase decisions for their businesses. Respondents who were not the primary decision-

makers were given an abbreviated version of the survey because they would be unlikely to have the

same knowledge of specific details related to lighting purchases as the decision-maker.

Ninety-one percent of respondents were the primary lighting purchase decision-makers, 5% said their

landlord or building owner is responsible for choosing and purchasing lighting, and the remaining 4%

said another external party is responsible.




10.5.3.3 General Population Survey Findings

Cadmus surveyed residential and small business customers about their awareness of the Efficient

Lighting Program, LED technology, energy efficiency in general, satisfaction with LEDs and various LED

traits (as well as the importance of LED traits to purchase decisions), and any changes in opinion about

PPL Electric Utilities after purchasing LED products discounted by PPL Electric Utilities.

Not all respondents answered every question in the surveys, so sample sizes vary by question or topic.

Awareness of Program, LEDs, and Energy Efficiency

Eighty-two percent of residential general population survey respondents (n=300) and 90% of small

business general population survey respondents (n=398) had heard of screw-in LED light bulbs,

consistent with PY8 results (80% and 88%, respectively).

Among residential respondents who had heard of screw-in LEDs, 90% had seen screw-in LEDs for sale in

stores. When asked to rate the influence of energy efficiency when making equipment purchase

decisions on a scale of 1 to 5, where 1 is not influential and 5 is extremely influential, small business

respondents gave an average rating of 4.1, with more than two-thirds giving ratings of 5 (45%) or 4

(24%).

Thirteen of 14 residential respondents who reported being aware of and having purchased LEDs through

a PPL Electric Utilities promotion or sponsored sale said the discounted price played a very important (8)

or somewhat important (5) role in their decision to purchase screw-in LEDs instead of some other type

of bulb. One respondent said the sale was neither important nor unimportant.

Among small business respondents (n=363), 39% knew PPL Electric Utilities provides funding for rebates

and discounts for energy-efficient light bulbs, and 25% (n=361) were aware that PPL Electric Utilities

offers discounted lighting products through participating distributors. The latter finding represents a

statistically significant increase relative to PY8 (19%).49 Nearly half (43%) of small business survey

respondents (n=330) in PY10 had seen PPL Electric Utilities marketing that explained the energy-saving

benefits of LEDs in the past year.

Of eight small business respondents who purchased LEDs through a PPL Electric Utilities promotion or

sponsored sale in PY10, five said the discounted price played a very important (2) or somewhat

important (3) role in their decision to purchase screw-in LEDs instead of some other type of bulb. Three

respondents said a promotion was neither important nor unimportant (1), not too important (1), or not

at all important (1).



Light Bulb Purchase Patterns

Cadmus asked residential customers to indicate whether they had recently purchased light bulbs in

general and, if so, how many of these were LEDs, CFLs, incandescent bulbs, halogens, and/or other bulb

types. The average light bulb purchaser bought 10.3 light bulbs during the previous six months, of which

more than two-thirds (roughly 6.6 per purchaser) were LEDs, as shown in Figure 10-1.

Figure 10-1. Light Bulb Purchase Patterns, by Technology – Residential

Source: Residential General Population Survey Question C3 (n=178): “Of these screw-in bulbs

purchased in the past 6 months, approximately how many were….”.

Of light bulbs purchased in the previous six months, 69% were LEDs. Sixty-two percent of LEDs

purchased by residential customers were purchased in retail stores.

Small business customers said they purchased light bulbs most frequently from retail stores (60%),

followed by electric distributors or suppliers (23%), as shown in Figure 10-2. Compared to PY8, the

percentage of small business customers who purchased LEDs at retail stores decreased, and the

percentage who purchased LEDs online increased. Both changes were statistically significant.50

50 Two-sample t-tests for the difference in proportions, statistically significant at 90% confidence.


Figure 10-2. Light Bulb Purchase Patterns, by Source – Small Business

Source: Small Business General Population Survey Question D3: “Where does your organization usually

purchase the majority of your light fixtures, lamps and controls?” Multiple responses allowed; percentages

add to more than 100%.The percentage in the box indicates the difference between PY10 and PY8 is

statistically significant at 90% confidence.

Satisfaction with LEDs

Cadmus asked questions about general satisfaction with LEDs.51 The results, indicating that both

residential and commercial customers were very satisfied with LEDs, are reported in this section.

Because respondents could skip questions if they did not want to answer them, not all respondents

provided an answer to every question. The number of respondents is indicated.

Nearly all residential general population respondents who have used LEDs (94%) were very satisfied or

somewhat satisfied with the screw-in LEDs they installed, as shown in Figure 10-3. Similarly, 91% of

small business LED purchasers were very satisfied or somewhat satisfied with the screw-in LEDs they

installed. Results were consistent with PY8, in which 93% of residential respondents and 92% of small

business respondents were either very satisfied or somewhat satisfied with their LEDs.

51 The survey did not ask satisfaction questions about the program because few participants realized they were participating in a program that offered upstream discounts.


Figure 10-3. PY10 Satisfaction with LEDs, by Sector

Source: Residential General Population Survey Question G1 (n=213), Small Business General Population

Survey Question J1 (n=187): “How satisfied, in general, were you with the screw-in LEDs you installed?”

Satisfaction with LED Traits

Both residential and commercial surveys investigated how satisfied LED users were with various bulb

traits. At least 74% of residential respondents were very satisfied or somewhat satisfied with every LED

trait shown in Figure 10-4. They were very satisfied most frequently with the longevity of LEDs (65%) and

least frequently with the cost of purchase (32%). LED brightness received the highest percentage of very

satisfied and somewhat satisfied responses, while the variety of LEDs available received the lowest

percentage. Among residential respondents, PY10 results were similar to PY8 results regarding

satisfaction with bulb traits.

Figure 10-4. PY10 Satisfaction with LED Traits – Residential

Source: Residential General Population Survey Questions G2a-g: “How satisfied were you with the screw-in

LEDs you installed in terms of…”


At least 73% of small business respondents were very satisfied or somewhat satisfied with the LED traits

shown in Figure 10-5. Unlike the residential results above, small business respondents were most

frequently very satisfied with the ease of finding LED products and LED brightness, and just 60% were

very satisfied with LED longevity (compared to 65% of residential respondents). Similar to residential

respondents, small business respondents rated LED brightness highest and variety lowest in total

satisfaction (very satisfied combined with somewhat satisfied responses), and cost received the lowest

percentage of very satisfied responses.

Figure 10-5. PY10 Satisfaction with LED Traits – Small Business

Source: Small Business General Population Survey Questions I2a-g: “How satisfied were you with the screw-

in LEDs you installed in terms of…”

As with the residential survey results, PY10 small business results were similar to PY8 results for

satisfaction with LEDs, albeit with slight (but not statistically significant) decreases for every trait.

Importance of LED Traits to Purchase Decisions

Cadmus asked residential survey respondents to rate the importance of four LED traits to their purchase

decisions. LED longevity was considered very important most frequently (65%) followed by cost (55%)

and energy use (55%). Manufacturer brand was generally considered not important, with five times as

many respondents choosing not at all important (35%) over very important (7%).

Figure 10-6 compares trends in the importance of LED traits by program year (PY10, PY8, and PY7). In

PY10, 65% of residential respondents mentioned longevity as very important, compared to 66% in PY8

and 71% in PY7.52 In PY10, 55% of respondents considered energy use as a very important trait, a



statistically significant decrease from 63% in PY8.53 Note that the spike in the importance of energy use

during PY8 differed from PY6 (57%) and PY7 (58%).54 Ultimately, longevity remains among the most

important factors for LED purchase decisions.

Figure 10-6. Importance of LED Traits to Purchase Decision, by Program Year – Residential

Source: Residential General Population Survey Question E1: “Please indicate the importance of…”

(PY10: n=300; PY8: n=300; PY7: n=337).

The percentage in the box indicates that the difference between PY10 and PY8

is statistically significant at 90% confidence.

Roughly one-third of all residential respondents (34%, n=300) provided open-ended responses to a

survey question about other factors they considered important to their LED purchase decisions (Figure

10-7). “Other” factors consist of responses mentioned by less than 5% of respondents, such as variety

and convenience, whether a bulb emits heat, bulb quality and safety, environmental safety, warranty,

and connectivity.




Figure 10-7. Other LED Traits Important to Purchase Decision – Residential

Source: Residential General Population Survey E2: “Are there any other factors that are important in your

decision when you buy light bulbs?” (n=300). Multiple responses allowed;

percentages sum to more than 100%.

Small business respondents most often said LED longevity was very important to their purchase decision

(61%), ahead of cost (50%) and energy use (48%). The importance of each trait has decreased since PY6,

which is statistically significant,55 but not since PY8. As with residential results, manufacturer brand

bears little importance to a small business customer’s decision to purchase LEDs. The results and trends

in Figure 10-8 strongly resemble the residential results shown in Figure 10-6.

Figure 10-8. Importance of LED Traits to Purchase Decision by Program Year – Small Business

Source: Small Business General Population Survey Question G1: “Please indicate the importance of…”

(PY10: n=361; PY8: n=269; PY7: n=385).



Roughly one-fourth of respondents (26%, n=361) said other factors were important to their LED

purchase decisions. As with the residential survey, most responses included factors such as size, shape,

style, availability of a warranty, safety, component quality, variety, brand, and packaging.


A detailed breakdown of Efficient Lighting Program finances and cost-effectiveness is presented in Table

10-12. TRC benefits were calculated using gross verified impacts. NPV PYTD benefits and costs are

expressed in PY10 dollars (PY10 includes months in both 2018 and 2019). NPV costs and benefits for

P3TD financials are expressed in the PY8 dollars. Participant costs are equal to the total of differences

between retail bulb prices and baseline bulb costs.56




56 Baseline bulb costs are from the Statewide Evaluator’s Incremental Cost database, version 3.1.



Table 10-12. Summary of Efficient Lighting Program Finances – Gross Verified

Row

# Cost Category PYTD ($1,000) P3TD ($1,000) (10)





EDC CSP EDC CSP



7 Marketing (4) - $253 - $732





12 NPV of increases in costs of natural gas (or other fuels) for fuel

switching programs - -

13 Total NPV TRC Costs (Net present value of sum of rows 4, 11,

and 12) (7) $7,441 $33,796





18 Total NPV TRC Benefits (Sum of rows 14 through 17) (8) (1) $50,185 $173,793

19 TRC Benefit-Cost Ratio (9) 6.74 5.14

(1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase.

These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and

technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits,

legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs,

including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at

marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.



Table 10-13. Summary of Efficient Lighting Program Finances – Net Verified






EDC CSP EDC EDC



7 Marketing (4) - $253 - $732








and 12) (7) $6,927 $30,869






19 TRC Benefit-Cost Ratio (9) 6.01 4.77 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase.



legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs,




10.7 Recommendations

Overall, the program has continued to exceed planned sales as LED prices continue to decline, adoption among customers increases, and all

bulbs sold through the program met current ENERGY STAR certification criteria. Program data are complete, consistent, and accurate, and PPL

Electric Utilities remains pleased with the implementation and performance of the program.

Recommendations are provided in Table 10-14, along with a summary of how PPL Electric Utilities plans to address the recommendations.

Conclusion 1: Reported savings for some

specialty bulbs would be more accurate (higher)

if the correct baselines for EISA-exempt bulbs

were used.

Findings Support

• As detailed in Section 10.2.2.1 Database Review, baseline wattages reported for some specialty bulb

types do not reflect the fact that these bulbs are exempt from the EISA ruling covering general

service lamps—that is, the reported baseline wattages used in the ex ante reported savings

calculations are too low.

Table 10-14. Status of Recommendations for Efficient Lighting

Efficient Lighting Program


EDC Status of Recommendation (Implemented,

Being Considered, Rejected and Explanation of

Action Taken by EDC)

Conclusion 1: Reported savings for some specialty

bulbs would be more accurate, (higher) if the

correct baselines for EISA-exempt bulbs were

used.

The ICSP could review the baseline wattage for EISA-exempt lamp

types to ensure they reflect the TRM baseline wattage for exempt

lamps and for reflector lamps and use either the applicable TRM

baseline wattage, the current federal standard for reflector lamps,

or the manufacturer-rated wattage equivalent for reflector lamp

types exempt from current federal standards.

Being considered.

The program encourages customers to save energy by sending them home energy reports that provide data about their energy use, a comparison of household energy use to similar customers in the same geographic area, and tips for saving energy (such as turning off lights and taking short showers) and product recommendations (such as LEDs, smart strips, and appliances).

HOME ENERGY EDUCATION PROGRAM


PY10 expenditures

72%

80%

of projected

of projected


$4,008

$1,540


PY10 saved

113,387 MWh/yr

42,829 MWh/yr

50%

104%

of projected

of projected


Satisfied with Home Energy Reports

66%


customers rec ived Home Energy Reports

PY10 PARTICIPATION

173,525

Chapter 11 Home Energy Education Program PPL Electric Utilities | 126

11 Home Energy Education The Home Energy Education Program encourages customers to save energy by sending them home

energy reports that provide data about their energy use, a comparison of household energy use to

similar customers in the same geographic area, and tips for saving energy (such as turning off lights and

taking short showers) and product recommendations (such as LEDs, power strips, and appliances). These

reports also guide customers to PPL Electric Utilities’ online energy management portal, My Ways To

Save,58 on which they can take a home energy assessment by entering detailed information about their

home and request a kit with energy-saving products.59

CLEAResult, the ICSP for all of PPL Electric Utilities’ residential programs, administered the Home Energy

Education Program and provided oversight of the home energy reports vendor. The ICSP subcontracted

with Tendril to develop and distribute the program’s educational offerings—the home energy reports,

online energy management portal, and online home energy assessments.

In PY10, the program mailed either one, five, or seven print home energy reports to customers,

depending on the cohort the customer belonged to. Customers with valid email addresses also received

these reports in electronic format and could ask to receive only the electronic reports. Customers could

also access the program’s energy management portal to set energy-saving goals, receive

recommendations toward reaching these goals, and check off any actions they had completed.

Customers who used the portal additionally received a weekly email with an energy saving tip and a

challenge, and customers could log into the portal to document their completion of the challenge.

The objectives of the Home Energy Education Program were these:60

• Encourage customers to adopt energy-efficient behaviors and install high-efficiency products

• Help customers become more aware of how their behavior and practices affect energy use

• Educate customers about no- and low-cost products and behavior changes that may reduce

energy consumption

• Educate customers about PPL Electric Utilities’ online resources

• Promote other PPL Electric Utilities energy efficiency programs

• Obtain participation of approximately 123,000 customers through 2021 with a total of

approximately 226,000 MWh/year gross verified savings

• Achieve high customer satisfaction with the program

58 PPL Electric Utilities’ Customer Engagement Hub (My Ways to Save) is a website with information about all available rebates. All PPL Electric Utilities’ customers have access to the hub and to the energy management portal; however, treatment group customers received specific encouragement through the home energy reports to visit the energy management portal.

59 The savings for the kits and energy-saving products are reported in the Energy Efficient Home Program.

60 Program objectives and targets are listed in PPL Electric Utilities’ revised EE&C Plan, November 2018.


The program operated as a randomized controlled trial where eligible customers were randomly

assigned to a treatment group or a control group. Customers in the treatment group received the home

energy reports. Treatment group customers who did not wish to receive the reports could opt out of the

program at any time. Customers in the control group did not receive the reports and were not told they

were part of the control group. The consumption data of control group customers provided the baseline

for estimating the savings from the home energy reports.

The same treatment and control group assignments from Phase II carried over into Phase III. The home

energy reports vendor identified new treatment and control group customers to expand the program

for Phase III, and Cadmus conducted the random assignments of these additional customers in early

PY8.

11.1 Participation and Reported Savings


A participant in the Home Energy Education Program is defined as a residential customer assigned to the

treatment group who received home energy reports. A participant who only received one report is

retained in the treatment group for analysis, even if the participant subsequently opted out.

The customer population is divided into six cohorts of customers known as “waves” defined by the dates

customers began receiving the home energy reports:

• Phase I Legacy Wave 1 received first report in PY2, April or May 2010

• Phase I Legacy Wave 2 received first report in PY3, June 2011

• Phase II Expansion Wave received first report in PY6, October or December 2014

• Phase II Low-Income Wave 1 received first report in PY6, October or December 2014

• Phase II Low-Income Wave 2 received first report in PY7, June 2015

• Phase III Expansion Wave received first report in PY8, June 2016

Cadmus evaluated the energy savings of all six waves.


Table 11-1 presents the participation counts, reported energy and demand savings for the Home Energy

Education Program in PY10 by customer segment. The count of participants is based on the number of

unique job numbers (referring to an account number for one household) in PPL Electric Utilities’ tracking

database. Cadmus’ impact analysis includes the number of customers who were intended to receive

treatment during PY10 including customers whose accounts became inactive during the year (195,669).


Table 11-1. PY10 Home Energy Education Program Participation and Reported Impacts

Parameter Residential Total ⁽¹⁾

PYTD # Participants (2) 173,525 173,525

PYRTD MWh/yr 42,079 42,079


PYVTD MWh/yr 42,829 42,829


PY10 Incentives ($1000) $0 $0

⁽¹⁾ Total may not match sum of columns due to rounding. (2) The participant count in PPL Electric Utilities’ tracking database is based on the number of unique job numbers (referring to an account number for one household) while the participant count for the impact analysis includes the number of customers who were intended to receive treatment during PY10 including customers whose accounts became inactive during the year (n=195,669).



The impact evaluation estimated the Home Energy Education Program’s energy savings. The program’s

experimental design and the large number of customers in the randomized treatment and control

groups allowed Cadmus to obtain accurate and precise estimates of the program’s savings. For each

wave, Cadmus conducted a panel regression analysis of individual monthly billing consumption for

customers in the treatment and control groups. Also, because the home energy reports encouraged

customers to participate in PPL Electric Utilities’ other energy efficiency programs, Cadmus estimated

energy savings from participation in these programs (see Appendix C.1.4 Uplift Analysis Methodology for

details). Cadmus subtracted the uplift savings from the residential portfolio savings to avoid double-

counting the uplift savings. (See section 11.4.2 Savings Uplift.)

The PY10 sampling strategy is summarized in Table 11-2. Cadmus included treatment group customers

in the regression analysis regardless of whether they received treatment (a home energy report)

because of the randomized experimental design, since no customers were intentionally left untreated in

PY10. The regression analysis therefore results in an estimate of the intent-to-treat effect. Only

customers with sufficient billing and tracking data were included in the regressions, but they were still

credited savings. Additional details about methodology and attrition are in Appendix C.1 Methodology.


Table 11-2. PY10 Home Energy Education Program Gross Impact Sample Design

Stratum

Population Size (1)


Design (2)

Achieved Sample Size (3) Impact Evaluation

Activity Treatment Control Treatment Control

Legacy Wave 1 50,000 50,000 N/A 47,792 47,793

Regression analysis on monthly billing data to estimate treatment effect (by stratum)

Legacy Wave 2 55,040 25,003 N/A 50,422 22,866

Expansion Wave 1 48,711 12,653 N/A 47,317 12,296

Low-Income Wave 1 73,500 18,560 N/A 71,752 18,106

Low-Income Wave 2 21,401 10,046 N/A 20,514 9,575

Phase III Expansion Wave 1 30,584 12,234 N/A 26,770 10,718

Program Total (4) 279,236 128,496 N/A 264,567 121,354

(1) Population size is based on the number of customers originally randomized prior to the start of the program.

(2) The population for each wave is based on data at the time of enrollment. Cadmus did not sample customers for inclusion in the analysis and therefore did not assume a proportion or Cv. (3) Cadmus included all customers in the analysis who had at least 11 months of pretreatment and at least one month of posttreatment billing data (details on attrition can be found in Appendix C.1 Methodology). These counts include customers whose accounts became inactive before PY10. (4) Total may not sum to all rows due to rounding.



Table 11-3. Home Energy Education Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 34,326 36,232 (1) 42,829 113,387 (2)

⁽1⁾ PY9 verified savings were reduced by 96 MWh/yr to conform with the SWE PY9 annual report findings. From PA PUC. SWE Annual Report Act 129 Program Year 9. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, Brightline Group, and Demand Side Analytics, LLC February 28, 2019. (2) Phase III verified savings may not match sum of program years due to rounding.

In PY10, the Home Energy Education Program reported energy savings of 42,079 MWh/yr, as shown in

Table 11-4, and demand reduction of 7.29 MW/yr, as shown in Table 11-5.


Table 11-4. PY10 Home Energy Education Program Gross Impact Results for Energy

Stratum PYRTD

MWh/yr Energy


Error Ratio

Relative Precision at 85%

C.L. PYVTD MWh/yr (1)

Legacy Wave 1 10,365 104% 0.04 5.39% 10,740

Legacy Wave 2 13,394 107% 0.05 7.63% 14,304

Expansion Wave 1 8,356 97% 0.09 13.29% 8,074

Low-Income Wave 1 4,807 98% 0.12 17.71% 4,725

Low-Income Wave 2 0 N/A 5.27 759.01% -33(2)

Phase III Expansion Wave 1 5,157 97% 0.08 11.64% 5,019

Program Total (3) 42,079 102% N/A 4.54% 42,829

(1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

(2) Cadmus calculated a negative treatment effect for treatment customers in Low-Income Wave 2, but the increase in energy consumption compared to the control group was statistically indistinguishable from 0.0 kWh/day.

(3) Total may not sum to all rows due to rounding.

Table 11-5. Home Energy Education Program Gross Impact Results for Demand



Error Ratio


C.L. PYVTD MW/yr (1)

Legacy Wave 1 1.79 101% 0.33 47.48% 1.81

Legacy Wave 2 2.31 104% 0.33 47.81% 2.41

Expansion Wave 1 1.45 94% 0.34 49.17% 1.36

Low-Income Wave 1 0.84 95% 0.35 50.69% 0.80

Low-Income Wave 2 0.00 N/A 5.56 800.05% -0.01

Phase III Expansion Wave 1 0.90 94% 0.34 48.70% 0.85

Program Total (2) 7.29 99% N/A 23.39% 7.23

(1) Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

(2) Total may not sum to all rows due to rounding.

The following factors affected the program’s achievements in PY10:

• Savings from the Phase III Expansion wave continued ramping up in PY10. The Phase III

Expansion Wave, launched in PY8, achieved savings of almost 1.3% in PY10, increasing from

0.3% in PY8 and 0.7% in PY9.

• The ICSP resumed treating low-propensity customers from whom they suspended treatment

in PY8 and most of PY9. The ICSP suspended treatment for some customers in Legacy Wave 1,

Legacy Wave 2, and Expansion Wave 3 who they determined had a low-propensity to save; the

assumption was that suspending treatment from these customers would not lower the average

daily savings for these waves overall since the low-propensity customers were not expected to


save energy.61 However, suspending treatment to low-propensity customers reduced the

number of treatment days Cadmus used to calculate program total ex post savings (see

Appendix C Home Energy Report Impact Evaluation Detail for savings algorithms), lowering both

realization rates and the percentage of projected savings achieved in PY8 and PY9. The ICSP

resumed treating low-propensity customers again at the end of PY9, and in PY10 Cadmus used

the full set of treatment days when calculating PY10 ex post savings.

The following factor led to variation between the reported and verified savings and to the observed

realization rates:

• Cadmus could not verify whether the ICSP sent treatment to some customers who were active

and eligible to receive treatment in PY10 but not flagged by the ICSP as receiving either

electronic or paper HERs during PY10. Cadmus kept these customers in the billing analysis when

estimating average daily savings per customer and calculated an “intent to treat” effect. Most of

these customers came from Legacy Wave 1 and Legacy Wave 2. Since the ICSP did not include

these customers in its ex ante savings, the realization rates were over 100%.


The Home Energy Education Program evaluation resulted in an estimate of net savings. The estimate

included any spillover that may have occurred within treated customer homes. No free ridership was

anticipated because customers did not choose to receive the home energy reports and no incentives

were provided. The evaluation did not estimate Home Energy Education Program gross savings;

therefore, the NTG ratio is irrelevant in this analysis.

11.4 Uplift Analysis

Cadmus estimated Home Energy Education Program uplift (the effect of the program on participation in

other PPL Electric Utilities efficiency programs) and the energy savings resulting from uplift in PY10.

Participation uplift savings appeared in the regression-based estimate of Home Energy Education

Program savings and the savings of any other PPL Electric Utilities efficiency programs that experienced

uplift. Therefore, Cadmus subtracted the Home Energy Education Program savings that were counted in

other programs from PPL Electric Utilities’ residential portfolio savings to avoid counting the savings

twice.

61 The ICSP’s subcontractor randomized customers with high propensities to save into the Phase III Expansion Wave to offset (and increase) savings lost from low-propensity customers, but customers in the Phase III Expansion Wave continually save less than low-propensity customers would have as part of the legacy waves because their pretreatment consumption is much lower.


The Phase III Evaluation Framework requires the estimation of home energy report savings attributable

to current and past efficiency program participation.62 For example, installation of a high-efficiency air

conditioner in PY4 is expected to yield savings in PY10 and through the expected life of the product. To

estimate the home energy report savings in PY10 that are attributable to the prior adoption of high-

efficiency air conditioners and other products, Cadmus collected historical energy efficiency program

data from PPL Electric Utilities’ tracking database. See Appendix C.1.4 Uplift Analysis Methodology for

details on participation uplift and uplift savings estimation methodology.

11.4.1 Participation Uplift

To estimate the effect of the Home Energy Education Program on participation in PPL Electric Utilities’

other efficiency programs, Cadmus compared the rates of participation between treatment and control

group customers in other Act 129 programs in PY10. Home energy reports had a positive average effect

on participation in other programs where rates of cross-program participation were greater for

treatment group customers.

Table 11-6 shows participation uplift results for PY10. On average, across all waves, treatment

customers participated in other PY10 programs at a rate 13% greater than control customers. Phase III

Expansion Wave customers had the greatest participation uplift compared to the other waves—on

average, treatment customers in this wave participated in other PPL Electric Utilities programs at a rate

16% greater than control customers. Participation uplift was positive for both legacy waves and the

Phase III Expansion wave, but was negative for Low-Income Wave 1. It is important to note that these

values are not cumulative across each waves’ existence. These values only reflect customers’ cross-

program participation starting in PY10.

Table 11-6. PY10 Home Energy Education Participation Uplift Summary

Wave

Control Group

Participation Rate

(per 1,000 Customers)

Participation Uplift

(Treatment Effect on

Participation Rate)

Percentage


Legacy Wave 1 29.4 1.2 4%

Legacy Wave 2 37.7 2.5 7%

Expansion Wave 1 33.0 0.4 1%

Low-Income Wave 1 67.3 -3.0 -5%

Low-Income Wave 2 57.0 3.4 6%

Phase III Expansion Wave 30.3 4.9 16%

Program Total 38.5 5.1 13%



11.4.2 Savings Uplift

Cadmus estimated savings uplift to determine whether treatment group customers saved more than

control group customers from cross-participation in other programs. Cadmus calculated savings uplift as

the difference in treatment and control groups’ average cross-program savings per customer, multiplied

by the number of treatment group customers. Savings uplift is positive if the per-customer savings

accrued in PY10 from current or previous participation in other Act 129 programs was greater for the

treatment group than for the control group. Cadmus accounted for the timing of product installations in

other programs and annualized their net savings using a weather-effects weight based on the product’s

end use.

Table 11-7 and Table 11-8 show energy and demand savings uplift results for PY10 resulting from PPL

Electric Utilities’ downstream programs. Across all waves, uplift accounted for 9% of Home Energy

Education Program energy savings and 8% of demand savings. Savings uplift decreased as a percentage

of program total savings from PY9, when 10% of Home Energy Education Program savings were

attributable to both HERs and cross participation in other programs, showing that in PY10, a greater

proportion of measured savings came from customers engaging in behavior changes and non-

incentivized equipment changes than in previous program years.

Low-Income Wave 2 did not achieve savings that were statistically distinguishable from 0 kWh. Although

they achieved negative energy savings uplift, meaning the average control customer saved more from

cross-program participation than the average treatment customer, the absolute value is quite small

(-9 MWh).

Table 11-7. PY10 Home Energy Education Downstream Uplift Energy Savings Summary

Wave

Average Cross-Program Savings per Customer (kWh/yr)

Total Uplift Savings

(MWh/yr)

Percent of Program Total

Savings Treatment

Group Control Group

Difference

Legacy Wave 1 353 329 23 761 7%

Legacy Wave 2 554 516 38 1,427 10%

Expansion Wave 1 357 347 10 384 5%

Low-Income Wave 1 384 377 7 311 7%

Low-Income Wave 2 276 277 -1 -9 26%

Phase III Expansion Wave 1 329 294 35 899 18%

Program Total (1) 393 367 26 3,773 9%



Table 11-8. PY10 Home Energy Education Downstream Uplift Demand Savings Summary

Wave Average Cross-Program Savings per Customer (kW/yr) Total Uplift

Savings (MW/yr)

Percent of Program Total

Savings Treatment Group Control Group Difference

Legacy Wave 1 0.063 0.060 0.003 0.11 6%

Legacy Wave 2 0.090 0.083 0.006 0.24 10%

Expansion Wave 1 0.054 0.053 0.002 0.07 5%

Low-Income Wave 1 0.050 0.048 0.002 0.08 10%

Low-Income Wave 2 0.036 0.037 0.000 0.00 83%

Phase III Expansion Wave 1 0.053 0.049 0.003 0.08 10%

Program Total (1) 0.060 0.059 0.001 0.58 8%


Cadmus estimated the Home Energy Education Program’s impact on upstream lighting (LED) purchases

by applying the default upstream lighting reduction factors from the Phase III Evaluation Framework, as

shown in Table 11-9 and Table 11-10.63 See Appendix C.1.4 Uplift Analysis Methodology for further

details on the uplift analysis for upstream lighting.

Table 11-9. PY10 Home Energy Education Upstream Lighting Uplift Energy Savings Summary

Stratum Population Verified

Savings (MWh/yr) (1)

Years in Program Reduction Factor Upstream Lighting Uplift (MWh/yr)

Legacy Wave 1 9,978 9 3.00% 299.35

Legacy Wave 2 12,877 8 3.00% 386.31

Expansion Wave 1 7,690 4.5 3.00% 230.70

Low-Income Wave 1 4,415 4.5 3.00% 132.44

Low-Income Wave 2 -25 4 3.00% -0.74

Phase III Expansion Wave 1 4,120 3 2.25% 92.71

Program Total (2) 39,056 N/A N/A 1,141 (1) Savings are adjusted to remove downstream uplift. (2) Total may not sum to all rows due to rounding.



Table 11-10. PY10 Home Energy Education Upstream Lighting Uplift Demand Savings Summary

Stratum Population Verified

Savings (MW/yr) (1)

Years in Program Reduction Factor Upstream Lighting

Uplift (MW/yr)

Legacy Wave 1 1.70 9 3.00% 0.05

Legacy Wave 2 2.17 8 3.00% 0.07

Expansion Wave 1 1.30 4.5 3.00% 0.04

Low-Income Wave 1 0.72 4.5 3.00% 0.02

Low-Income Wave 2 0.00 4 3.00% 0.00

Phase III Expansion Wave 1 0.76 3 2.25% 0.02

Program Total (2) 6.65 N/A N/A 0.19

(1) Savings were adjusted to remove downstream uplift shown in Table 10-8. (2) Total may not sum to all rows due to rounding.

Additionally, Cadmus deducted 1,305 MWh/yr and 0.14 MW/yr from the residential sector to account

for the 45,000 LED bulbs that PPL Electric Utilities mailed to high-energy use customers in the low-

income waves’ treatment groups, distributed through the Phase II Residential Retail Program. PPL

Electric Utilities claimed savings from these bulbs in PY7. Cadmus deducted these savings from the

residential sector because the savings were counted in the residential Home Energy Education program

in PY10 (there is no separate low-income program that claimed these savings in PY10).

In total, Cadmus deducted Home Energy Education Program uplift savings of 6,218 MWh/yr and 0.91

MW/yr from the residential portfolio savings. The total energy uplift savings are 15% of the total gross

savings compared to 16% of PY9 gross energy savings (5,921 MWh/yr). Cadmus deducted program uplift

savings from the residential portfolio as opposed to the Home Energy Education Program savings

because uplift savings are attributable to this program as well as the other programs for which Cadmus

verified savings.



demand savings estimates to calculate the verified savings estimates for the Home Energy Education

Program in PY10. Because the NTG ratio is irrelevant in this analysis, net savings are the same as verified

gross savings.


Table 11-11. PYTD and P3TD Home Energy Education Program Savings Summary

Savings Type Energy (MWh/yr) (1) Demand (MW/yr) (1)



PYVTD Net (2) (3) 42,829 7.23

P3RTD Gross 116,422 68.22

P3VTD Gross 113,387 20.09

P3VTD Net (2) (3) 113,387 20.09

(1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target. (3) The NTG ratio is irrelevant; net savings are the same as verified gross savings. (4) Cadmus estimated a 12% demand realization rate in PY8, leading to large discrepancies in P3RTD and P3VTD demand savings.



The process evaluation of the Home Energy Education Program focused on assessing customer

satisfaction with the home energy reports and the reports’ influence on energy-saving practices.


The PY10 process evaluation activities for the Home Energy Education Program included these:


• A customer survey with a stratified random sample of the treatment group (telephone and

online)

The PY10 process evaluation activities were consistent with the evaluation plan except Cadmus did not

review the logic model. Cadmus reviewed the logic model in PY8 and PY9 and decided not to conduct a

logic model review for PY10 because the PY10 program theory and logic did not change from PY8 and

PY9. Table 11-12 lists the process evaluation sampling strategy.


Table 11-12. Process Evaluation Sampling Strategy

Stratum Stratum


Size (1) Mode


Design

Target Sample

Size

Achieved Sample

Size

Records Selected

for Sample

Frame (2)




Staff 2 Telephone in-depth interview

N/A 2 2 2 100%

Customer Surveys with Treatment Group

Phase I Legacy Waves

83,605

Telephone survey

0.5 50 53 26,575 98%

Online survey

0.5 50 50 11,977 40%

Phase II Expansion Wave

46,578

Telephone survey

0.5 50 50 14,878 96%

Online survey

0.5 50 51 6,784 100%

Phase III Expansion Wave

30,584

Telephone survey

0.5 50 50 10,470 94%

Online survey

0.5 50 53 4,482 100%

Phase II Low-Income Wave 1

69,354

Telephone survey

0.5 50 50 14,650 100%

Online survey

0.5 50 39 5,126 100%

Phase II Low-Income Wave 2

20,868

Telephone survey

0.5 50 16 2,719 100%

Online survey

0.5 50 4 1,050 100%

Program Total 250,991 N/A N/A 500 416 98,711 N/A (1) Population size is the total number of treatment group customers who received a home energy report in PY10 at the time of the survey activity. These numbers may not match those reported in the impact analysis sections of this report due to different time periods.

(2) Sample frame is a list of participants with contact information who have a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities tracking database. After selecting all unique records, Cadmus removed any records from the population if the customer had participated in a survey in the last three months, was selected for another program survey, did not have valid contact information (email or telephone number), was on the do not call list, or opted out of the online survey. (3) Percent contacted means the percentage of the sample frame contacted to complete surveys.


Cadmus completed a phone and online survey with 416 treatment group customers, using a stratified

random sampling approach by treatment wave. The survey did not achieve its overall target sample size

of 500 completes because of difficulty obtaining enough respondents from the two low-income waves,

which had the fewest number of records available. Table 11-12 contains the final number of completed

surveys by strata, which produced a measurement of program satisfaction with ±3% precision at 90%

confidence. The population estimates are weighted averages of the stratum level results.


Additional details about Cadmus’ approach to contacting customers, sample attrition, and survey

weighting methodology are presented in Appendix I.1.2 Survey Approach and Appendix R Survey

Methodology.


In February and March of 2019, Cadmus interviewed the program managers from PPL Electric Utilities,

the ICSP, and the home energy reports vendor. The interviews focused on identifying and assessing

changes to program design and delivery from PY9 to PY10 and learning what areas are working well and

about any possible challenges.


This section discusses program delivery, satisfaction, customer engagement, and energy-saving practice

adoption. The treatment group customer demographics are found in Appendix I.1.1 Additional Findings.


Treatment group customers in all six waves received print and/or electronic home energy reports in

PY10,64 and the number of reports varied by wave. As shown in Table 11-13, the primary change to the

program from PY9 was to the frequency of reports. PPL Electric Utilities and the ICSP decided to change

report frequency because a key finding from the PY9 evaluation was that the two oldest waves

produced higher savings but had lower satisfaction and the newest waves produced lower savings but

had higher satisfaction. Therefore, to optimize savings and satisfaction, the program reduced the

number of reports for the two oldest waves and increased the number of reports for the newer waves.

The evaluation found that savings increased from 36,328 MWh/yr in PY9 to 42,829 MWh/yr in PY10, but

customer satisfaction with the home energy reports did not improve from PY9 (65%) to PY10 (66%). PPL

Electric Utilities and the ICSP plan to reduce the number of reports for the two oldest waves in PY11 by

one report (i.e., four print and/or electronic reports in PY11).

64 The print and electronic home energy reports were identical in content.


Table 11-13. Home Energy Report Delivery Frequency

Wave (Launch Year) PY8 PY9 PY10

Phase I Legacy Wave 1 (2010) 6 print reports and/or

6 electronic reports

6 print reports and/or




Phase I Legacy Wave 2 (2011) 6 print reports and/or






Phase II Expansion Wave 1 (2014) 6 print reports and/or






Phase II Low-Income Wave 1 (2014) no reports sent 1 electronic report 1 print report and


Phase II Low-Income Wave 2 (2015) no reports sent 1 electronic report 1 print report and


Phase III Expansion Wave 1 (2016) 6 print reports and/or






PPL Electric Utilities and the ICSP also followed the PY9 evaluation recommendation to conduct A/B

message tests that focused on improving customer satisfaction and reducing customer frustration with

the similar homes comparison. During PY10, the home energy reports vendor, the ICSP’s subcontractor,

conducted A/B message tests with 13,500 treatment customers. Customers assigned to group A

received the homes comparison message similar to prior years, and customers assigned to group B

received a new message. Both groups received the message twice in the electronic home energy

reports, one in September and one in November 2018.

A survey conducted by the ICSP’s subcontractor found that the new message “bothered customers less”

and had “a positive impact on satisfaction.” The ICSP’s subcontractor plans to apply the insights from

the message tests to the PY11 home energy reports.

Cadmus also analyzed the differences in satisfaction with the home energy reports between group A and

group B customers using the evaluation’s PY10 survey respondents. Of the 13,500 A/B message test

customers, 64 respondents, all from the Phase II Expansion Wave, answered the report satisfaction

question in Cadmus’ PY10 survey conducted in April 2019. These respondents were evenly split between

group A (n=32) and group B (n=32). Both groups met the minimum sample size threshold of 30 for

statistical testing. A significantly higher proportion of group B respondents (66%) than group A

respondents (38%) were satisfied with the report.65 This suggests that among the Phase II Expansion

Wave customers, the new home comparison message may contribute to customer satisfaction. Cadmus

did not have survey responses from treatment customers in other waves.



11.6.3.2 Satisfaction with Home Energy Reports

In PY10, 66% of treatment group respondents said they were satisfied with the home energy reports, as

shown in Figure 11-1. This was similar to PY9 where 65% of respondents said they were satisfied. The

percentage of not at all satisfied respondents significantly decreased from 8% in PY9 to 5% in PY10.66 Of

the 40 respondents who said they were not too satisfied or not at all satisfied, Cadmus asked a

follow-up question about their reasons. Respondents most often complained about the similar homes

comparison (52%) and did not believe the home energy reports had accurate data (27%).

Figure 11-1. Overall Satisfaction with Home Energy Reports

The percentage in the white box (in PY10) indicates that the difference between PY10 and PY9 is statistically significant, p≤0.10.

Survey data were weighted by wave and indicated by the notation “nw.” The analysis includes everyone who answered the

program satisfaction question even if they did not complete the entire survey.

Source: Survey question, “How would you rate your overall satisfaction with the Home Energy Reports?”

Similar to PY9, the newer waves were more satisfied with the home energy reports than the older

waves. As shown in Figure 11-2, the Phase III Expansion Wave had a significantly greater proportion of

very satisfied and somewhat satisfied respondents than the Phase I Legacy Waves and Phase II

Expansion Wave.67 This pattern suggests possible report fatigue among the customers who have been

receiving the home energy reports the longest. Satisfaction had an inverse relationship to savings; Phase

I Legacy Waves had the highest percentage of daily savings (1.3% to 1.9%) while Phase II Expansion

Wave (1.2%) and Phase III Expansion Wave (1.2%) had lower daily savings in PY10.




Figure 11-2. Overall Satisfaction with Home Energy Reports by Wave

The percentage in the white box indicates that Phase III Expansion Wave and Low-Income Wave 1 are significantly different.

Survey data were not weighted. The analysis includes everyone who answered the program satisfaction question even if they

did not complete the entire survey. The sample size for Low-Income Wave 2 was too small for statistical significance testing.

Source: Survey question, “How would you rate your overall satisfaction with the Home Energy Reports?”

11.6.3.3 Suggested Report Improvements

Cadmus asked respondents to name one thing that PPL Electric Utilities could change to improve the

home energy reports, and 105 respondents gave suggestions. The most frequent suggestions were

these:

• Improve the similar homes comparison by providing a more clear explanation or making the

comparison fair (31%)

• Improve the accuracy of the report by factoring in more details or making it easier for customers

to make corrections (17%)

These suggestions were consistent with the reasons cited by customers who were dissatisfied with the

reports.

11.6.3.4 Customer Engagement

This section describes the findings on report readership, reception to the reports, awareness of energy

efficiency programs, and online visits.

Readership of and Reception to the Home Energy Reports

The majority of treatment group respondents read the print and electronic home energy reports.

Specifically, 94% of respondents said they read or skimmed the print reports, and 89% said they read or

skimmed the electronic reports. Readership levels did not significantly differ between print and

electronic. Figure 11-3 shows the readership level of the home energy reports.


However, significantly more Phase III Expansion Wave respondents said they read the print report

thoroughly (47%, n=92) compared to Phase I Legacy Wave respondents (35%, n=100) and Phase II

Expansion Wave respondents (33%, n=88).68

Figure 11-3. PY10 Readership of Home Energy Reports

Survey data were weighted by wave.

Source: Survey question, “Which of the following best describes what you did with

the last Home Energy Report you received in the mail/by email?”

The survey asked treatment group respondents to agree or disagree with seven statements about the

home energy reports using a 10-point scale, where 0 meant strongly disagree and 10 meant strongly

agree. As shown in Figure 11-4, respondents most often strongly agreed with statements about

comprehension and content of the reports. Respondents least often strongly agreed with statements

about motivating the household and the accuracy of the similar homes comparison.

Figure 11-4. Agreement Level to Statements about the Home Energy Reports


Source: Survey question, “To what extent do you agree or disagree with the following statements about the Home Energy

Reports? Use a 0 to 10 scale where 10 means strongly agree and 0 means strongly disagree.”



Awareness of Energy Efficiency Programs

Most treatment group respondents said they were familiar with energy efficiency programs and rebates

from PPL Electric Utilities. Figure 11-5 shows that 72% of respondents said they were familiar, with most

indicating that they were somewhat familiar (54%). Notably, at the wave level, significantly more Phase I

Legacy Wave respondents said they were familiar (79%, n=103) than did Phase II Expansion Wave

respondents (68%, n=701) and Low-Income Wave 1 respondents (66%, n=88).69 This finding reflects the

fact that Phase I Legacy Wave respondents have been receiving the home energy reports the longest

and therefore have had the most exposure to programs and rebates.

Figure 11-5. Familiarity with Energy Efficiency Programs


Source: Survey question, “How familiar are you with PPL Electric Utilities' energy efficiency and

conservation programs to help you with ways to use energy more efficiently?”

The home energy reports promoted the following PPL Electric Utilities energy efficiency programs and

rebates from one to four times during PY10:

• Online assessment (four times)

• Lighting (three times)

• Appliance recycling (three times)

• Central air conditioner (one time)

• Ductless heat pump (one time)

• Heat pump water heater (one time)

• Online marketplace (one time)

• Refrigerator rebate (one time)

• Smart thermostat (one time)

Respondents who said they were very familiar, somewhat familiar, or not too familiar (n=374) were

asked to name a program or rebate. Eighty-seven percent of respondents could name or describe a PPL

Electric Utilities program or rebate. The respondents most frequently mentioned appliance rebates

(16%), the Appliance Recycling Program (13%), and lighting (8%). These results somewhat aligned with

the frequency of promotion in the home energy reports. Even though the online assessment was



promoted four times in the home energy reports during PY10, no respondents mentioned it. This may

be because customers do not think of the online assessment as a program.

Online Visits

Figure 11-6 shows that 46% of treatment group respondents visited PPL Electric Utilities’ website in the

past six months. The survey followed up with these respondents by asking if they had visited the section

called My Ways To Save, the energy management portal, and 48% said they had visited the portal.

Figure 11-6. Self-Reported Visits to Utility Website and Energy Management Portal

Note: Survey data were weighted by wave.

Source: Survey questions, “Have you visited the PPL Electric Utilities' website in the past 6 months?”

and “On the PPL Electric Utilities' website, have you ever visited the section called Ways to Save?”

The home energy reports encouraged customers to take the online assessment and enter detailed

information about their home. Completing the assessment would improve the data accuracy in the

reports and the similar homes comparison. Cadmus reviewed the number of treatment group customers

who completed the online assessment component of the Energy Efficient Home Program in PY10 and

found that no customer completed it.

11.6.3.5 Self-Reported Adoption of Energy-Saving Practices

The reports provided customers with personalized energy-saving behavioral tips and product

recommendations. Treatment group respondents were asked whether they had adopted any of the

energy-saving behaviors and made any of the home improvements listed in Figure 11-7. The top

behavior changes and improvements were associated with lighting (turn off lights in unoccupied rooms,

98%, and installed LEDs, 87%).

Respondents reported adopting energy-saving behaviors more often than making the home

improvements. A majority of respondents reported adopting five of the seven energy-saving behaviors,

while a majority of respondents reported making only one of the nine home improvements. This trend

may be because making home improvements often requires a financial investment.


Figure 11-7. Self-Reported Adoption of Energy-Saving Practices

Survey data were weighted by wave. Percentages add up to more than 100% because multiple answers were allowed. Some

behaviors and home improvements were not applicable to respondents, and this lowered the n.

Source: Survey questions, “For each item, please answer yes if you do this or no if you do not do this”

and “Please answer yes or no whether you have done this in the past 12 months.”

The financial investment explanation is further supported by the significant differences observed at the

wave level. Significantly more Low-Income Wave 1 respondents than Phase I Legacy Wave respondents

reported adopting the four behaviors shown in Table 11-14. A consistent pattern did not emerge for

home improvements at the wave level.


Table 11-14. Significant Behavioral Differences between Low-Income Wave 1 and Phase I Legacy Wave Respondents

Energy-Saving Behavior Low-Income

Wave 1 Respondents

Phase I Legacy Wave

Respondents

Statistical Significance

Level

Keep water heating temperature at or below 120 degrees 73% (n=85) 59% (n=99) p≤0.05

Keep the heating thermostat to 68 degrees or lower in winter 68% (n=87) 55% (n=101) p≤0.10

Unplug electronics or appliances when not in use 54% (n=89) 36% (n=103) p≤0.05

Keep the air conditioning thermostat to 78 degrees or higher in summer

36% (n=76) 24% (n=100) p≤0.10


A detailed breakdown of program finances and cost-effectiveness is presented in Table 11-15. Cadmus

calculated TRC benefits using gross verified impacts. The net present value program year to date (NPV

PYTD) benefits and costs are expressed in PY10 dollars (PY10 includes months in both 2018 and 2019).

Net present value costs and benefits for P3VTD financials are expressed in PY8 dollars. Net verified

savings are equal to gross verified savings because the program is assumed to have a NTG ratio of 1.0.


Table 11-15. Summary of Home Energy Education Program Finances–Gross and Net Verified


1 EDC Incentives to Participants - -


3 Participant Costs (net of incentives/rebates paid by utilities) - -

4 Incremental Measure Costs (Sum of rows 1 through 3) (1) - -

EDC ICSP EDC ICSP



7 Marketing (4) - $247 - $553








and 12) (7) $1,540 $3,682


15 Total NPV Lifetime Electric Capacity Benefits $507 $1,141

16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits - -

17 Total NPV Lifetime Non-Electric Benefits (Fossil Fuel, Water) - -


19 TRC Benefit-Cost Ratio (9) 1.62 1.46 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio-level costs and are assigned to customer sectors at the end of the phase.


technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs. (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits,

legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs,





Overall, the Home Energy Education Program achieved 104% of its projected savings and observed 66% customer satisfaction with the home

energy reports in PY10. Recommendations are provided in Table 11-16 along with a summary of how PPL Electric Utilities plans to address the

recommendations.

Conclusion 1: Savings are following expected trends, given waves’ sizes and length of historical treatment.

Findings Support

• The legacy waves’ savings continued to persist. Despite not treating the low-propensity customers for much of

PY9 and treating them for all of PY10, the legacy waves’ savings showed little change. (See Appendix C.1.3.)

• Low-Income Wave 1 savings increased slightly from PY9, after not receiving treatment for much of PY8. (See

Appendix C.1.3.)

• The Phase III Expansion Wave savings continued to ramp up to savings comparable to Phase I Legacy Wave 1 and

Phase II Expansion Wave 1. (See Appendix C.1.3.)

Conclusion 2: The two oldest waves may not need as many home energy reports to achieve optimal savings.

Findings Support

• In PY10, the program reduced the number of reports for the two oldest waves (Phase I Legacy Wave 1 and Legacy

Wave II). These waves received five print and/or electronic reports in PY10. The program plans to reduce the

number of reports again for the two oldest waves in PY11. These waves are scheduled to receive four print

and/or electronic reports PY11. (See section 11.6.3.1 Program Delivery.)

• For the past three years, the two oldest waves have had the highest savings for the program and have plateaued

at 1.5% to 1.9%, even when the number of reports was reduced this past year. (See Appendix C.1.3.)

Conclusion 3: Customer satisfaction with the home energy reports did not improve from PY9 after changing the number of reports delivered. Impacts from this change may take some time to see.

Findings Support

• The program reduced the number of reports for the two oldest waves and increased the number of reports for

the other newer waves to optimize savings and satisfaction. (See section 11.6.3.1 Program Delivery.)

• In PY10, 66% of treatment group respondents said they were satisfied with the home energy reports. PY10

remained similar to PY9 (65%). Phase III Expansion Wave had a significantly greater proportion of satisfied

respondents than Phase I Legacy Waves and Phase II wave, suggesting possible report fatigue among the

customers who have been receiving the reports the longest. (See section 11.6.3.2 Satisfaction with Home Energy

Reports.)


• Treatment group respondents most often complained about the similar homes comparison (52%) and did not

believe the home energy reports had accurate data (27%). (See section 11.6.3.2 Satisfaction with Home Energy

Reports.)

• The home energy reports vendor conducted A/B message tests focused on improving customer satisfaction and

reducing customer frustration with the similar homes comparison. The vendor plans to apply the insights from

the tests to the PY11 home energy reports. (See section 11.6.3.1 Program Delivery.)

Conclusion 4: Low-Income Wave 2 is unlikely to generate statistically significant savings in the future.

Findings Support

• In PY9, the evaluation could not distinguish Low-Income Wave 2 savings from 0.0 kWh/day because of the small

number of customers in the wave. As a result, the PY9 evaluation recommended the discontinuation of the wave.

The program plans to discontinue treatment for Low-Income Wave 2 in PY11 and PY12. (See Appendix C.1.3.)

• Low-Income Wave 2 did not generate statistically significant savings in PY10. (See Appendix C.1.3.)

Conclusion 5: The home energy reports continued to deliver cross-program participation and savings, except for the online assessments.

Findings Support

• The home energy reports promoted energy efficiency programs and rebates from PPL Electric Utilities one to four

times during PY10. (See section 11.6.3.4 Customer Engagement.)

• Total energy uplift increased from PY9 (5,921 MWh/yr) to PY10 (6,218 MWh/yr), however total uplift as a

percentage of gross savings decreased from PY9 (10%) to PY10 (9%). (See section 11.4.2 Savings Uplift.)

• Most treatment group respondents (72%) said they were familiar with energy efficiency programs and rebates

from PPL Electric Utilities. (See section 11.6.3.4 Customer Engagement.)

• Even though the online assessment was promoted four times in the home energy reports during PY10, no

respondents mentioned it when asked to name a program or rebate. (See section 11.6.3.4 Customer

Engagement.)

• No treatment group customer completed the online assessment in PY10. (See section 11.6.3.4 Customer

Engagement.)


Table 11-16. Status of Recommendations for the Home Energy Education Program

Home Energy Education Program





Conclusion 1: Savings are following expected trends, given

waves’ sizes and length of historical treatment. Consider conducting an experiment to test whether

sending two reports vs. four reports would be the optimal

number of reports for the two oldest waves.

Being considered. Conclusion 2: The oldest waves may not need as many

home energy reports to achieve optimal savings.

Conclusion 3: Customer satisfaction with the home energy

reports did not improve from PY9 after changing the

number of reports delivered. Impacts from this change may

take some time to see.

Continue delivering a reduced number of home energy

reports to customers and apply A/B message test insights

to the reports as early as possible.

Being considered.

Consider investigating the demographic and segment

characteristics of the customers who are dissatisfied and/or

disengaged with the reports. Use the collected information

to identify opportunities for piloting a new behavioral

product or service.

Being considered.

Conclusion 4: Low-Income Wave 2 is unlikely to generate

statistically significant savings in the future.

Continue plans to stop sending home energy reports to

Low-Income Wave 2 in PY11 and PY12. Implemented.

Conclusion 5: The home energy reports continued to

deliver cross-program participation and savings, except for

the online assessments.

Continue promoting other PPL programs in the home

energy reports and consider new ways to engage

customers to complete the online assessment

Being considered.

W RELIEF ASSISTANCE PROGRAM (WRAP)The program offers products and services to income-qualified customers to help reduce their electric consumption.


PY10 expenditures

$22,086

$8,702



PY10 saved

36,172 MWh/yr



79%of projected



93% satisfied with the quality of work provided by WRAP auditor

93% satisfied with quality of energy efficiency products

94% satisfied with the ease of program enrollment

4,666


Manufactured homes (all job types)

Full-cost job

Low-cost jobs

Baseload jobs6,671

11,621

PY10 PARTICIPATION

57%of projected

Multifamily buildings (all job types)53

95% satisfied with the energy education received

86%of projected

LED Giveaway (packs)2,450

Chapter 12 Winter Relief Assistance Program PPL Electric Utilities | 152

12 Winter Relief Assistance Program

The Act 129 Winter Relief Assistance Program (WRAP), also known as Low-Income WRAP, operates in

parallel with PPL Electric Utilities’ Universal Services Programs’ Low-Income Usage Reduction Program

(USP LIURP) WRAP. Both programs are designed to reduce electric consumption for low-income

customers.

PPL Electric Utilities offers services to income-qualified customers residing in single-family homes,

master-metered multifamily units, individually metered multifamily units, and manufactured homes.70

WRAP is delivered by CMC Energy, the ICSP, which is responsible for outreach, customer recruitment,

audits, education, and the direct installation of equipment in customers’ homes. The ICSP also operates

a customer call center and supports marketing and tracking activities. The ICSP uses qualified

community-based organizations (CBOs) and contractors for tasks, including the installation of energy-

savings products and services and replacement of outdated and inefficient equipment with program-

qualifying energy-efficient equipment. CMC Energy, along with Franklin Energy and York Home

Performance, subcontractors to the ICSP, are responsible for targeted outreach and recruitment for the

manufactured homes located in manufactured or mobile home parks.

All qualifying customers receive a free energy audit that evaluates their home for eligible energy-saving

products. The home energy auditor refers to a preapproved list of products and services along with

criteria to determine if appliances and other large equipment can be replaced cost-effectively, within

the program’s budget (program acquisition cost and total funding). For all qualifying customers, PPL

Electric Utilities offers direct installation of a range of energy efficiency products and services,71

including HVAC, lighting, weatherization, water saving and heating, appliances, appliance recycling, and

home health and safety. WRAP also offers energy education delivered by auditors who make

recommendations to encourage customers to conserve energy.

Through WRAP, PPL Electric Utilities provides four types of service (also known as job types) at no cost

to the income-qualified customer. These services include baseload (offered to customers without

electric heat and without an electric water heater), low-cost (offered to customers without electric heat

but with electrically heated water), full-cost (offered to customers with electric heat), and an initiative

offering services to targeted manufactured home park customers.

70 Individually metered low-income multifamily residences are eligible for the same improvements as individually metered single-family low-income residences under WRAP. Furthermore, individually metered manufactured homes are eligible for the same improvements as any other type of individually metered home receiving services from WRAP.

71 PPL Electric Utilities eliminated refrigerators, heat pump water heaters, and window air conditioners from the program in February 2018.


The objectives of WRAP are these:72

• Provide low-income customers with an array of no-cost energy-saving equipment, products and

education to help reduce their energy costs

• Increase the health and safety of low-income customers’ homes by installing no-cost measures

such as smoke and carbon monoxide detectors, which may be coordinated with or implemented

by USP LIURP

• Achieve high customer and trade ally satisfaction through high-quality service and an impactful

program offering

• Promote other PPL Electric Utilities low-income energy efficiency and assistance programs

• Achieve a total approximate reduction in energy use of 50,000 MWh/year gross verified savings



An Act 129 WRAP participant is defined as a PPL Electric Utilities customer who lives in an income-

eligible household (150% of the federal poverty level) and receives a WRAP audit. Each treated

household (single-family or multifamily) is identified in the PPL Electric Utilities’ tracking database with a

unique billing account number.

Each master-metered multifamily building has a unique billing account number. As shown in Table 12-1,

WRAP participation counts each master-metered multifamily building as a single participant. The

population size for the WRAP gross impact sample design counts individual tenant units in master-

metered multifamily buildings, as shown in Table 12-2.

In PY10, the ICSP organized LED giveaway community events for the first time in Phase III. During these

community events, a pack of LED bulbs was given away to each participating household (the number of

bulbs included in each pack varied by LED giveaway community event). A total of 2,450 packs of LED

bulbs (total of 5,800 LED bulbs) were given away in four community events. Each participating

household received one pack of LED bulbs from the ICSP. Cadmus calculated the total number of

participants for LED giveaway community events by the total number of packs of LED bulbs distributed

in LED giveaway community events. See Appendix J.1.1 Methodology for details about participation

counts for LED giveaway events.

72 Program objectives are listed in PPL Electric Utilities’ revised EE&C Plan (Docket No. M-2015-2515642), November 2018.




payments for WRAP in PY10 by customer segment. (See Appendix J.1.4 Records Review Findings for

additional discussion about participant counts.)

Table 12-1. WRAP Participation and Reported Impacts

Parameter Government/

Nonprofit/ Education (GNE)

Low-Income Small C&I Total (1)

PYTD # Participants 42 (2) 15,408 (3) 12 (4) 15,462 (5)

PYRTD MWh/yr 1,333 19,559 258 21,151

PYRTD MW/yr 0.10 1.91 0.02 2.03

PYVTD MWh/yr 1,187 17,681 230 19,097

PYVTD MW/yr 0.10 1.78 0.02 1.90

PY10 Incentives ($1,000) $0 $0 $0 $0 (1) Total may not match sum of columns due to rounding. (2) GNE category consists of 42 participants, including 41 master-metered multifamily buildings and one WRAP baseload job in an individually metered multifamily building. (3) Low-Income category consists of 12,958 WRAP jobs (Baseload, Low-Cost, Full-Cost and Manufactured Home Initiative) and 2,450 packs of LED bulbs distributed in LED giveaway community events. (4) Small C&I category consists of 12 master-metered multifamily buildings. (5) Total number for participants counts each master-metered multifamily building as a single participant

based on the participant definition.



Cadmus conducted the activities described below to evaluate the WRAP gross impacts. See Appendix

J.1 Gross Impact Evaluation for details on these activities.

• Database review. Cadmus reviewed all records in PPL Electric Utilities’ tracking database and

compared these to the records in the participant data (ERMS) provided by the ICSP. Cadmus

verified discrepancies with the ICSP prior to conducting any analyses.

• Audit records review. Cadmus reviewed a random sample of ICSP’s home-audit records for the

five strata listed in Table 12-2. Reviews of audit records completed by the home energy auditors

at the job site involved verifying reported quantities and relevant inputs for savings calculations.

Cadmus verified all data fields in the audit records against the PPL Electric Utilities’ tracking

database (e.g., home address, water heater fuel type, heating fuel type, reported quantities, and

baseline conditions).


• Engineering analysis. Cadmus conducted an engineering analysis for the five strata listed in

Table 12-2 and used the findings from the audit records review as inputs to the engineering

algorithms from the PA TRM.73

• Participant Phone Surveys. Cadmus conducted phone surveys with a random sample of

program participants to verify that products were installed as reported and to collect supporting

data to analyze the impact of energy education for WRAP participants.


In PY10, Cadmus collaborated with PPL Electric Utilities and the ICSP to collect the required data to

verify energy savings and demand reduction for WRAP. Cadmus obtained the ICSP’s Energy Reduction

Management System (ERMS) database extract for verification and assessment of participant records.

The ICSP also provided audit records quarterly for a random sample of sites. In addition, Cadmus

conducted a phone survey with a sample of program participants to verify that products were installed

as reported.


The sample design consisted of five strata: one for each job type (baseload, low-cost, and full-cost), one

for master-metered multifamily units, and one for manufactured home participants. This strategy

allowed for an examination of savings by stratum. Within each stratum, Cadmus applied a simple

random sampling method to select a sample of homes for verification.

Each master-metered multifamily building is a single participant based on the participant definition for

WRAP; however, the sample population size for the master-metered multifamily strata counts individual

tenant units in master-metered multifamily buildings as Cadmus performed home audit records review

and engineering analyses for individual tenant units in master-metered multifamily buildings. The

sampling strategy is summarized in Table 12-2.

73 Pennsylvania Public Utility Commission. Technical Reference Manual. June 2016.


Table 12-2. PY10 WRAP Gross Impact Evaluation Sample Design

Stratum Population

Size

Assumed Proportion


Target Sample Size


Impact Evaluation

Activity

Baseload 6,671

0.5

24 24

Records review and

engineering analysis

Low-Cost 4,666 24 24

Full-Cost 1 (1) N/A N/A

Manufactured Home Initiative (all job types)

1,621 24 24

Master-Metered Multifamily (all job types)

2,267 (2) 24 24

Program Total 15,226 (3) 96 96

(1) Since there was only one full-cost job in PY10, Cadmus applied the full-cost stratum’s realization rate from PY9. (2) 53 master-metered multifamily buildings (41 under GNE category and 12 under Small C&I category) that participated in WRAP in PY10 have 2,267 tenant units and therefore 2,267 jobs were added to the total number of jobs. Cadmus performed home audit records review and engineering analysis for individual tenant units in master-metered multifamily buildings. Therefore, the population size for the WRAP gross impact sample design counts individual tenant units in master-metered multifamily buildings individually. See Appendix J.1.3 Participant Counts for additional discussion about participant counts. (3) 2,450 packs of LED bulbs distributed in LED giveaway community events are not part of the PY10 WRAP gross impact sample design. See Appendix J.1.1 Methodology for details about energy savings verification for LED giveaway events.


Table 12-3 shows the program’s verified gross savings.

Table 12-3. Winter Relief Assistance Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 2,652 14,423 (1) 19,097 36,172 (2)

(1) PY9 verified includes PY8 reported savings verified in PY9. (2) Phase III verified savings may not match sum of program years due to rounding.

In PY10, WRAP reported energy savings of 21,151 MWh/yr, as shown in Table 12-4. In PY10, WRAP

achieved a program energy realization rate of 90%, weighted by stratum. Table 12-5 shows demand

reduction of 2.03 MW/yr. Both tables are shown by program stratum.


Table 12-4. PY10 WRAP Gross Impact Results for Energy

Stratum PYRTD

MWh/yr

Energy Realization

Rate


Relative Precision

at 85% C.L.

PYVTD MWh/yr

Baseload 8,496 89% 0.13 3.81% 7,532

Low-Cost 7,824 91% 0.08 2.29% 7,082

Full-Cost 2 141% (1) N/A N/A 3


3,052 94% 0.07 2.20% 2,872


1,590 89% 0.07 2.11% 1,416

LED Giveaway 187 103% 0.00 0.00% 193

Program Total (2) 21,151 90% N/A 1.70% 19,097

(1) Since there was only one full-cost job in PY10, Cadmus applied the full-cost stratum’s realization rate from PY9. (2) Total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Table 12-5. PY10 WRAP Gross Impact Results for Demand


Realization Rate Sample Cv or Error Ratio (2)

Relative Precision

at 85% C.L.

PYVTD MW/yr

Baseload 0.84 91% 0.11 3.26% 0.76

Low-Cost 0.75 94% 0.11 3.29% 0.71

Full-Cost 0.00 93% N/A N/A 0.00


0.30 95% 0.07 2.01% 0.29


0.12 98% 0.04 1.35% 0.12

LED Giveaway 0.02 103% 0.00 0.00% 0.02

Program Total (1) 2.03 93% N/A 1.76% 1.90 (1) Total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Two primary factors contributed to the level of savings achieved in PY10:

• The ICSP achieved high program participation in the second half of PY9 and this positive trend

continued throughout PY10. Because of improved program participation in PY10, WRAP

achieved 19,097 MWh/yr of verified energy savings in PY10. In comparison, the program verified

energy savings in PY9 and PY8 was 14,412 MWh/yr and 2,663 MWh/yr, respectively.

• The ICSP improved the program delivery in the second half of PY9 and continued to apply these

best practices throughout PY10. Improved program delivery throughout PY10 helped WRAP to

achieve a program energy realization rate of 90% in PY10. In comparison, the program energy

realization rate in PY9 and PY8 was 82% and 76%, respectively.

Cadmus identified the following factors that led to differences between reported and verified savings

and the overall realization rate for WRAP in PY10:

• The most important factor driving the program’s realization rate overall is energy education.

The ICSP reported ex ante savings of 160 kWh/yr for every participant. Cadmus’ estimated


energy education savings for PY10 was 89 kWh/yr. This is an improvement compared to the

average energy education savings in PY9 of 54 kWh/yr.

• Differences in reported and evaluated ISRs for six products (LEDs, LED nightlights, efficient

showerheads, power strips, and kitchen and bathroom aerators) also contributed to the

difference in reported and verified savings. Evaluated ISRs for the program in PY10 remain high

compared to PY8, and they are about the same as PY9. Evaluated ISRs for the program in PY10

are generally close to 100% (e.g., 100% for LEDs, 95% for aerators and showerheads, and 85%

for smart strips) and there is little room for improvement for most products.

• The last factor contributing to the difference between reported and verified savings are a

confluence of minor differences between the reported data, record reviews, and calculation

differences for a variety of products. These instances were rare and the overall effect on the

realization rate was small (see Appendix J.1.4 Records Review Findings for additional details).


WRAP is offered to income-eligible customers in the low-income community, and the work is completed

at no cost to the customer. No free riders are anticipated among participants. That is, income-

constrained customers are not likely to purchase the energy efficiency products on their own. A NTG

ratio of 1.0 is appropriate for this program. Therefore, the evaluation did not estimate net savings.


Cadmus applied PY10 realization rates to the reported energy and demand savings estimates to

calculate the verified savings estimates for WRAP in PY10 (Table 12-6). Because the NTG ratio is 1.0, net

savings are the same as verified gross savings.

Table 12-6. PYTD and P3TD WRAP Savings Summary




PYVTD Net (2) (3) 19,097 1.90

P3RTD Gross 42,171 4.13

P3VTD Gross 36,172 3.82

P3VTD Net (2) (3) 36,172 3.82 (1) Total may not match sum of rows due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target. (3) Cadmus assumed there is no free ridership in this low-income program. Therefore, no net savings analyses were conducted.




Cadmus conducted the PY10 process evaluation with a focus on program delivery and participation and

addressed the following research objectives:

Identify areas of program success

• Identify areas that may benefit from

program improvements

• Assess satisfaction with multifamily

building property managers

• Assess customer satisfaction with overall

experience and installed products


The PY10 process evaluation for the WRAP included these activities:



• Telephone participant surveys

• Interviews with master-metered

multifamily building property managers

• Logic model review

Table 12-7 lists the process evaluation sampling strategy.


Cadmus conducted telephone surveys with participants to assess program satisfaction of participants,

assess energy education savings, and verify product installation. Cadmus administered the first wave of

telephone surveys (n=80) in February 2019 and the second wave of telephone surveys (n=75) in July

2019. Both surveys asked identical questions. Cadmus created a stratified random sample to target 59

Baseload Job participants, 48 Low-Cost Job participants, and 48 Manufactured Home Park participants

across the two waves for a total of 155 survey respondents. This sampling plan achieved a confidence

and precision level of 90/10. The sample frame excluded any participant who completed a survey in the

past three months or requested not to be contacted.

Completed participant surveys produced a measurement of program satisfaction with ±10% precision at

90% confidence. In PY10, Cadmus achieved a 5% response rate among 3,322 phone records attempted,

which is slightly higher than PY9—a 4% response rate among 3,430 phone records attempted (see

Appendix J.2.2 Survey Approach for attrition and Appendix R Survey Methodology for phone survey

methodology).


In July 2019, Cadmus conducted three interviews with WRAP managers from PPL Electric Utilities and

the ICSP. The interviews focused on changes made to the program in PY10 from PY9, the planned

changes for PY11, program successes, and program challenges.


Table 12-7. WRAP Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size

Achieved Sample

Size

Number of Records

Selected for Sample

Frame (1)


Achieve Sample (2)


Key Individuals from PPL Electric Utilities and ICSP

Telephone In-depth Interview

3 N/A 3 3 3 100%

Market Actor Interviews

Master-Metered Multifamily Property Managers

Telephone In-depth Interview

13(3) N/A 4 3(4) 13 100%

Participant

Program Participants (Baseload, Low Cost & Manufactured Home Park Initiative)

Telephone Survey

11,423 0.5 155(5) 155 8,335 40%

Program Total

11,439 161

(1) Sample frame is a list of participants with contact information who have a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities database at the time of the surveys. After selecting all unique records, Cadmus removed any records from the population if the customers had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (email or telephone number), were on the do not call list, or opted out of the online survey. (2) Percent contacted means the percentage of the sample frame called to complete surveys. (3) A total of 13 property managers managed all of the master-metered multifamily buildings that participated in WRAP in PY10. The ICSP provided contact information for all 13 property managers to Cadmus. (4) The three property managers who Cadmus interviewed manage 11 master-metered multifamily properties. (5) Cadmus developed the target sample size of 155 participants to achieve ±10% precision at 90% confidence for each wave of surveys. Cadmus conducted the first wave of participant phone surveys in February, 2019 (n=80) and the second wave in July, 2019 (n=75).

12.5.2.3 Master-Metered Multifamily Building Property Manager Interviews

In PY10, Cadmus conducted interviews with three of the 13 property managers who oversaw master-

metered multifamily buildings and participated in WRAP in PY10. The ICSP provided Cadmus with the

contact information for all 13 property managers. Prioritizing property managers who managed multiple

master-metered multifamily buildings in PY10, Cadmus reached out to each property manager at least

three times by email or phone. Cadmus completed interviews with three participating property

managers who managed a total of 11 master-metered multifamily buildings.

The objectives of these interviews were to understand how well PPL Electric Utilities delivered the

program, assess resident awareness and interaction with the WRAP contractor staff, ascertain the

managers’ satisfaction with PPL Electric Utilities and the program overall, and solicit suggestions for

improvement.


12.5.2.4 Logic Model Review

In PY10, Cadmus reviewed the logic model to determine whether the program was implemented as

designed and/or evolved from the Phase III EE&C plan.74 Cadmus reviewed, but did not update, the logic

model since the program is operating as expected and is on track to meet the anticipated intermediate

and long-term outcomes (see Appendix J.2.1 Additional Findings for logic model review details).



Overall, Cadmus found that the ICSP continued to smoothly deliver WRAP in PY10, with both PPL Electric

Utilities and the ICSP reporting effective delivery of the program throughout PY10.

After the ICSP made substantial changes to WRAP in the second half of PY9, it did not make major

changes to program delivery in PY10. The ICSP enrolled income-qualified participants, completed audits,

installed energy-saving products, and served clients as expected. In PY10, two teams (the ICSP along

with subcontractor Franklin Energy) completed 1,621 WRAP manufactured home initiative jobs. In

comparison, in PY9 and PY8, one team (Franklin Energy) completed 963 and 57 manufactured home

initiative jobs, respectively. PPL Electric Utilities reported that the time frame for completing the

average WRAP job decreased by about 50% at the end of PY9, which remained consistent throughout

PY10.

Program Changes and Improvements

There were no major changes to WRAP in PY10. The positive actions taken by the ICSP in the second half

of PY9 continued throughout PY10—improved communication with clients, scheduling practices, and

knowledge of the program logistics—all contributed to completing jobs in a shorter time frame and

increasing number of completed jobs. Overall, this improved realization rates and improved customer

satisfaction in PY10.

In PY10, program stakeholders made one change to WRAP:

• LED giveaway events. The ICSP organized four community events in PY10. The community

events were planned with organizations that provide service to low-income customers and

those in need. The ICSP and PPL Electric Utilities provided co-branded event marketing

materials. The ICSP sent invitations to PPL Electric Utilities’ OnTrack75 low-income customers for

three out of four community events. PPL Electric Utilities and ICSP staffed the events and

provided LED bulbs, WRAP information, energy education activities, and prizes during these

74 PPL Electric Utilities’ revised EE&C Plan (Docket No. M-2015-2515642), November 2018.

75 OnTrack is a special payment program for low-income households at or below 150% of the federal poverty level who have trouble paying the full cost of their electric bill. OnTrack is PPL Electric Utilities’ Customer Assistance Program (CAP) and is one of its Universal Service Programs (USP). This program, funded by residential customers and administered by community-based organizations, started as a pilot in 1993.


community events. (See Appendix J.1.1 Methodology for additional information about LED

giveaway events.)

In PY11, PPL Electric Utilities and the ICSP plan to add activities to serve the low-income community.

These include sending home energy reports to low-income customers, with tips to save energy tailored

to this customer segment, and providing student energy education and energy conservation kits to

students in schools identified with a high low-income population.

Marketing and Outreach

The marketing and outreach methods described by the ICSP and PPL Electric Utilities for PY10 remained

consistent with methods from the second half of PY9 (which included improved design and content for

the marketing materials). However, PPL Electric Utilities and the ICSP marketed the program to

customers via email more often in PY10.

Cadmus’ telephone surveys asked program participants’ opinion about the best way for PPL Electric

Utilities to inform them about energy efficiency programs and rebates. Forty-nine percent (n=150) said

bill insert or newsletter, 29% said email from PPL Electric Utilities; 6% said through the PPL Electric

Utilities website, 5% said through social media such as Twitter, Facebook, or Instagram; and 3% said

through a home energy auditor. Significantly fewer PY10 participants than PY9 participants (58%; n=217)

said bill insert or letter, but the other results remained consistent across program years (p<0.10).76

One of the three master-metered multifamily property managers interviewed learned about WRAP from

the ICSP program staff, and two learned about WRAP from the owner of the property. All three property

managers preferred to be informed about PPL Electric Utilities’ programs through email.

When asked who made the decision to participate in WRAP, two out of three property managers said

that the property owner made the decision, showing that outreach to key individuals in property

management can be an effective method to increase program participation. One property manager who

learned of the program through the ICSP needed approval from the U.S. Department of Housing and

Urban Development since the building was already under an energy service performance contract.

Each of the three property managers notified tenants about WRAP via direct mail.

Energy Education

The WRAP contractors provided some one-on-one education to residents during their visits to install the

energy efficiency products. At the end of the PY9, the ICSP started mailing quarterly reminder postcards

that list the energy-education recommendations. Cadmus found that quarterly reminder postcards

helped program participants to remember the recommendations and therefore more survey

respondents reported following some of the home energy auditor’s recommendations to save energy.

Unlike previous program years, none of the property managers interviewed organized an energy-

76 Cadmus used a two-tailed t-test to test for statistical significance between PY10 and PY9 responses at the 90% confidence level (p<0.10)


education seminar for their building residents, and each reported that the ICSP subcontractor did not

offer to provide an energy-education seminar to building residents due to low participation.

12.5.3.2 Satisfaction


Overall, Cadmus found that 92% of survey respondents were either very satisfied (78%) or somewhat

satisfied (14%) with their overall program experience, which is consistent with the satisfaction ratings

from PY9 (Figure 12-1).

Figure 12-1. Participant Satisfaction with WRAP

Source: "Thinking about your overall experience with the PPL Electric Utilities WRAP program,

how would you rate your satisfaction?”

Cadmus found that respondents were generally satisfied with various program components in PY10

(Figure 12-2). Significantly more PY10 respondents (78%; n=153) than PY9 respondents (70%; n=228)

were very satisfied with the ease of program enrollment at the 90% confidence level (p≤.10). Similarly,

significantly more PY10 respondents (78%; n=152) than PY9 respondents (70%, n=224) were very

satisfied with the energy education they received from the WRAP home energy auditor (p≤0.10).


Figure 12-2. Participant Satisfaction with Different Program Components

Source: Survey Question “Please indicate how satisfied you are with each of the following components…”

Property Manager Satisfaction

Master-metered multifamily property managers were asked about their satisfaction with WRAP overall

and with specific elements of the program. All three property managers were satisfied with the

program, with two very satisfied and one somewhat satisfied.

All three were either very or somewhat satisfied with the WRAP contractors’ interaction with tenants in

their building and the overall quality of the work. None of the respondents said that the ICSP conducted

an in-depth energy audit in their building tenant units; instead, they said that the ICSP’s contractor just

assessed how many LEDs, showerheads, and smart strips to bring back at a later date.

Property managers were somewhat satisfied or not too satisfied with the performance of the

showerheads, LEDs, and advanced power strips. They provided the following concerns with various

program elements:

• The program’s showerhead requirements do not meet the needs of disabled tenants. The

program requires the showerheads be installed directly into a wall but disabled tenants need

the showerheads installed on an adjustable slide bar so they can sit while showering.

• The advanced power strips with motion detectors were not appropriate for all elderly or

disabled tenants. One property manager reported that residents were dissatisfied when their

TVs turned off after a period of inactivity.

• LEDs did not live up to all the property managers expectations for long-lasting bulbs. Two of

three property managers said approximately 12 LEDs failed within one month of installation.

Energy Education Satisfaction

Survey respondents were asked if the energy education provided by the WRAP home energy auditor

helped them understand ways to save energy in their home. In PY10, significantly more customers (94%;


n=116) found the energy education to be very useful (66%) or somewhat useful (28%) than did the

customers in PY9 (83%; n=198) (p≤.10).

Cadmus found that achieving energy savings attributable to energy education improved in PY10 over

PY9. Significantly more PY10 respondents (75%; n=155) than PY9 respondents (65%; n=227) said the

home energy auditor provided recommendations about ways to save electricity and reduce energy costs

while visiting the participant’s home (p≤.10). When asked if they could list some of the

recommendations that the home energy auditor provided, 75% (n=115) listed a few recommendations.

Table 12-8 lists the percentage of respondents who remembered a recommended behavior.

Table 12-8. Recalled Recommended Behaviors

Recommended Behavior Percentage of Respondents

Recommended Behavior

Percentage of Respondents

Turn off lights when not in use or not home

28% Wash clothes in cold water 19%

Adjust thermostat settings or use ceiling fans

28% Manage and cut back water use 13%

Turn off or unplug appliances and/or electronics when not in use

23% Use a surge protector or power strip for electronics

13%

Purchase efficient bulbs 20% Apply weatherstripping around windows and doors

7%

Turn down water heater temperature 20% Other1 15%

Source: Survey question, “Can you tell me some of the recommendations that were provided to you during the visit?” Multiple responses allowed. (n=115) 1 Other included purchasing energy-efficient appliances, closing the basement door so heat does not escape,

adding foam insulation, and wrapping pipes.

When asked if they had followed up on any of the recommendations, significantly more PY10

respondents (91%; n=116) than PY9 respondents (57%; n=180) said yes (p≤.10). Of those 116

respondents, a majority (92) were able to list specific recommendations that they took.

Suggested Program Improvements

Although PY10 survey respondents reported high levels of satisfaction with the program, Cadmus

identified some challenges that can be addressed to improve education provided by contractors,

customers’ satisfaction with some of the products, and the level of contractor professionalism and

training.

Although 72% (n=155) of survey respondents said that there was nothing their WRAP home energy

auditor could have done to improve their program experience, 67% did provide suggestions (n=30).

When asked what PPL Electric Utilities could change about the program to improve it, WRAP

participants made 48 suggestions:

• Provide more education about program or products installed (n=11)

• Expand program eligible items or services (i.e., refrigerators, weather-stripping installation for

elderly) (n=11)

• Ensure auditors provide a professional service and increase their training (n=10)


• Decrease the processing time for sign-up applications (n=6)

• Increase marketing for the program (n=4)

• Improve the quality of installed items (i.e., faucet aerators, night lights) (n=3)

• Follow up on installed equipment to ensure quality and functionality (n=3)

Each of the three property managers interviewed made suggestions to improve the program. One

recommended that the program remove the advanced power strip, since the product does not work

well for the immobile elderly population, and another recommended that the program allow

contractors to install showerheads on a sliding bar to accommodate the needs of elderly or disabled

tenants. The third recommended that the program expand its product line to include tubular LED lights.


A detailed breakdown of program finances and cost-effectiveness is presented in Table 12-9. TRC

benefits were calculated using gross verified impacts. NPV PYTD costs and benefits are expressed in

PY10 dollars (PY10 includes months in both 2018 and 2019). NPV costs and benefits for P3TD financials

are expressed in PY8 dollars. Net verified savings are equal to gross verified savings because the

program is assumed to have a NTG ratio of 1.0.






Table 12-9. Summary of WRAP Finances–Gross and Net Verified

Row #

Cost Category PYTD ($1,000) P3TD ($1,000) (10)





EDC CSP EDC CSP



7 Marketing (4) - $288 - $850






- -


$8,702 $20,231






19 TRC Benefit-Cost Ratio (9) 1.14 0.77 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.



Overall, the Winter Relief Assistance Program has performed according to the program design, is administered well, and has exceeded both its

estimated participation and planned savings. The ICSP has consistently made improvements to program delivery over the last two program years

and WRAP achieved a higher energy realization rate in PY10 than in PY9 and PY8. WRAP is currently on target to meet its projected Phase III

savings and the savings for the low-income sector as a whole are on target as well. Satisfaction with the program was very high, with 92% of

participants reporting they were very or somewhat satisfied with the program. Recommendations are provided in Table 12-10, along with a

summary of how PPL Electric Utilities plans to address the recommendations.

Conclusion 1: The ICSP is on pace to

deliver the program and low-income

sector savings with the planned PY11 and

PY12 activities.

Findings Support

• In PY10, the ICSP delivered services to a total of 15,462 WRAP participants and exceeded the planning

estimates of 7,000 WRAP participants per year. (See section 12.1.2 Program Participation and Reported

Impacts.)

• The ICSP sent reminder postcards to participants about ways to save energy, which boosted energy

education savings. (See section 12.5.3.1 Program Delivery.)

• WRAP achieved 30% of the Phase III savings (19,097 MWh/yr) in PY10 alone and is currently on target to

meet its projected Phase III energy savings. (See section 12.2.2 Gross Impact Evaluation Results.)

• PPL Electric Utilities and the ICSP plan to add activities to serve the low-income community in PY11. These

include sending home energy reports to low-income customers, with tips to save energy tailored to this

customer segment, and providing student energy education and energy conservation kits to students in

schools identified with a high low-income population. (See section 12.5.3.1 Program Delivery.)

Conclusion 2: The ICSP’s improvements to

program delivery over the last two

program years led WRAP to achieve a

higher energy realization rate in PY10 than

in PY9 and PY8. The realization rate for the

current mix of improvements may plateau

without revising reported energy

education savings.

Findings Support

• PY10 WRAP energy realization rate (90%) was higher than the energy realization rates achieved in PY9

(82%) and PY8 (76%). (See section 12.2.2 Gross Impact Evaluation Results.)

• Cadmus found the ISRs to be high relative to PY8 and PY9. Most products have ISRs around 100%--only

Tier 2 advanced power strips had ISRs below 90%. (See Appendix J.1.6 In-Service Rates.)

• The ICSP made improvements to the energy education delivery by sending quarterly reminder postcards

to participants (see section 12.5.3.1 Program Delivery). Cadmus verified the energy education savings as

89 kWh/yr per household in PY10 (see section 12.2.2 Gross Impact Evaluation Results), the highest energy

education savings since the beginning of Phase III. However, the ICSP reports savings of 160 kWh/yr,

resulting in a 55% realization rate.


Conclusion 3: Improved program delivery

in PY10 helped the overall program

satisfaction reach 92%. Minor adjustments

to support elderly or disabled participants

may further increase overall program

satisfaction.

Findings Support

• Cadmus found that 92% of survey respondents were satisfied with their overall program experience. (See

section 12.5.3.2 Evaluation Activities.)

• Some program participants and property managers indicated that Tier 2 smart strips are not appropriate

for elderly or disabled customers. These customers were often dissatisfied with Tier 2 smart strips as their

TVs turned off after a period of inactivity. (See section 12.5.3.2 Evaluation Activities.)

• Some program participants and property managers indicated that the program’s showerhead

requirements do not meet the needs of disabled tenants. The program requires the showerheads be

installed directly into a wall but disabled tenants need the showerheads installed on an adjustable slide

bar so they can sit while showering. (See section 12.5.3.2 Evaluation Activities.)

• Some program participants and property managers indicated that auditors could provide more detailed

energy-efficiency recommendations other than the products installed. (See section 12.5.3.2 Evaluation

Activities.)

Table 12-10. Status of Recommendations for WRAP





EDC)

Conclusion 1: The ICSP is on pace to deliver the program and low-income sector savings with the planned PY11 and PY12 activities.

To achieve the energy savings projected for the low-income sector in Phase III, continue delivering the level of services as planned.

Implemented.

Conclusion 2: The ICSP’s improvements to program delivery over the last two program years led WRAP to achieve a higher energy realization rate in PY10 than in PY9 and PY8. The realization rate for the current mix of improvements may plateau without revising reported energy education savings.

To increase program energy realization rates over the 90% mark, maintain the high-quality program delivery achieved in PY10. Consider revising reported (ex ante) energy education savings to 89 kWh/yr, the most recent evaluated savings for energy education.

Being considered.






EDC)

Conclusion 3: Improved program delivery in PY10 helped the overall program satisfaction reach 92%. Minor adjustments to support elderly or disabled participants may further increase overall program satisfaction.

To increase program satisfaction over the 92% mark, consider three revisions: (1) instructing contractors to consistently provide energy efficiency recommendations, (2) work with master-metered property managers to determine if their buildings have a high percentage of elderly or disabled residents and do not provide Tier 2 smart strips to elderly or disabled customers or ask these customers if they would like the smart strip, and (3) adapt program showerhead requirements to meet the needs of disabled tenants.

Being considered.

The program offers a wide range of energy efficient products, rebates, education, and services that give customers a variety of customizable solutions to increase their home’s energy efficiency.

ENERGY EFFICIENT HOME PROGRAM


PY10 expenditures

$17,608

$6,529



PY10 saved

46,407 MWh/yr

17,661 MWh/yr

63%

123%

of projected

of projected


98%of projected

104%of projected



83%

satisfied with the clarity of application requirements81%

satisfied with rebates forqualifying equipment and services79%

satisfied with the information about how to save energy

A total of 19,863 received rebates for:

PY10 PARTICIPATION

Online and in-home auditand energy conservation kits7,852

Weatherization765

New homes1,002

10,244 Efficient equipment

Chapter 13 Energy Efficient Home Program PPL Electric Utilities | 172

13 Energy Efficient Home Program

The Energy Efficient Home Program is designed for new construction and existing homes. The program

offers a wide range of energy-efficient products, rebates, education, and services that give customers a

variety of customizable solutions to increase their home’s energy efficiency. The program has six

components: New Homes (new home construction incentives), Audit and Kits (in-home energy audits

and online home energy assessments, both of which include energy-savings kits), Weatherization,

Efficient Equipment, and the Online Marketplace (a pilot offering new in PY10).

New Homes. In PY10, the New Homes component offered up to $2,500 in incentives for the

construction of energy-efficient new homes through either $0.30 per annual kWh/yr saved for homes at

least 15% above the residential building code (2009 IECC) or $0.35 per annual kWh saved for ENERGY

STAR®-rated homes at least 15% above code.

Audit and Kits. The Audit and Kits component offers in-home energy audits and the online home energy

assessments and provides tools and information that help residential customers make decisions about

actions they can take to improve the energy efficiency of their homes. Energy savings accrue from the

kit of low-cost energy-efficient products mailed to the customers. The kits currently contain LEDs, faucet

aerators, energy-efficient showerheads, pipe insulation, and weatherstripping. Faucet aerators and

showerheads are only distributed to homes with electric water heating. In-home audit customers can

also receive a water heater setback.

Weatherization. The Weatherization component provides rebates to customers who make any of these

three eligible home improvements: ceiling insulation (minimum of R11 above code), wall insulation

(minimum of R11 above code), or air sealing.

Efficient Equipment. This component offers rebates for eligible products, including air source heat

pumps (SEER 16+), ductless heat pumps (< 5.4 tons, ≥ SEER 16, ≥ HSPF 8.6), central air conditioners

(SEER 16+), heat pump water heaters (≥ 2.3 EF), efficient pool pumps (variable speed drive), ENERGY

STAR or ENERGY STAR Most Efficient refrigerators and dehumidifiers, advanced smart thermostats,

ASHP tune-ups, duct sealing, and fuel-switching to non-electric ENERGY STAR water heaters or high-

efficiency central heating equipment (natural gas or propane furnace [AFUE 95], oil furnace [AFUE 85],

or fossil fuel boiler [AFUE 85]).

Online Marketplace. This pilot launched in PY10 and is managed by a third party. The Online Market

component is a web-based storefront that allows qualified customers to order energy-efficient products

online, submit inquiries via e-mail, and view educational materials. Customers must have a PPL Electric

Utilities account number to shop. Incentives are applied directly to the energy-efficient products, but

customers can see the pre-incentive price as well. The marketplace offers products that differ

seasonally, such as weather stripping, rope caulk, LED light bulbs, LED holiday light strings advanced

power strips, occupancy sensor switches, smart thermostats, and dehumidifiers.

PPL Electric Utilities’ energy efficiency program staff provide overall strategic direction and program

management. The EM&V staff oversee evaluation activities and coordinate with program staff.


CLEAResult, the ICSP, manages the program and delivers the Audit and Kit, Weatherization, and Efficient

Equipment components of the program to customers. This involves maintaining a call and rebate

processing center, conducting in-home audits, recruiting and educating trade allies, and marketing the

program to achieve sufficient participation. Performance Systems Development (PSD) is a subcontractor

to the ICSP and is responsible for the program’s New Home component. PSD processes applications and

assists builders and Home Energy Rating System (HERS) raters. The Online Marketplace pilot is managed

by the Energy Federation, Inc. (EFI), a subcontractor to the ICSP.

In PY10, the objectives of the Energy Efficient Home Program were these:78

• Encourage customers to view energy

efficiency in a holistic manner

• Educate construction industry

professionals and other trade allies about

the benefits of energy-efficient homes

• Promote the construction of energy-

efficient new homes

• Provide customers with education, audits,

surveys, and energy-saving solutions

• Reduce energy consumption by

approximately 73,000 MWh/year in gross

verified savings

• Achieve high customer and trade ally

satisfaction with the program



For all components of the Energy Efficient Home Program, a participant is defined as a rebated project,

and each project is assigned a unique job number in the program tracking data. For the New Homes

component, a participant is defined as the single-family home or a tenant unit in a newly constructed

multifamily building.79



payments for the Energy Efficient Home Program in PY10, by customer segment.

78 Program objectives are listed in PPL Electric Utilities’ revised EE&C Plan (Docket No. 2015-2515642), November 2018.

79 In PY10, New Homes participation is estimated at 80% single-family homes and 20% multifamily units, based on the percentages found in the 40 records reviewed.


Table 13-1. Energy Efficient Home Program Participation and Reported Impacts


PYTD # Participants 23 1 19,773 69 19,866

PYRTD MWh/yr 34 0 20,256 143 20,434

PYRTD MW/yr 0.01 0.00 3.57 0.02 3.59

PYVTD MWh/yr 31 0 17,492 138 17,661

PYVTD MW/yr 0.01 0.00 3.12 0.02 3.14

PY10 Incentives ($1000) $1 $0 $3,182 $6 $3,189




Cadmus conducted these research activities to inform the gross impact evaluation:

• Program database review

• Records reviews of participant rebate

applications and supporting

documentation

• Participant surveys

• REM/Rate modeling review for New

Homes80

Cadmus conducted a database review of each program component to ensure that appropriate data

were collected and to confirm that ex ante savings were properly calculated using the appropriate PA

TRM algorithms. Cadmus also reviewed a sample of records to evaluate the savings impacts of the Audit

and Kit, Weatherization, and Efficient Equipment program components. The records reviews

accomplished the following:

• Verified that product types were correctly categorized based on the verified installed products

• Verified that reported equipment data in PPL Electric Utilities’ tracking database matched

information from rebate applications, Air Conditioning, Heating, and Refrigeration Institute

(AHRI) certificates, invoices, and other supporting documentation

• Calculated ex post savings using the PA TRM algorithms and verified equipment data

Cadmus used the results of online participant surveys from PY10 to calculate the ISR for kit products

delivered through the online assessment component of the Audit and Kit stratum. Cadmus relied on the

ISRs calculated for the online assessment component for kits distributed through the in-home audit

component as well. For all other products, Cadmus used ISRs from surveys conducted in PY8.

80 More information about REM/Rate software and applications is available online: http://www.remrate.com/

http://www.remrate.com/


For the New Homes component, Cadmus reviewed a sample of 40 REM/Rate models, reviewed their

HERS rater documentation, and conducted engineering analyses to verify the energy and demand

savings. Cadmus used these results to determine the ex post savings.

Cadmus did not conduct site visits in PY10. To supplement the data used to verify energy and demand

savings, Cadmus also referred to site visit findings, realization rates, and ISRs from PY8.


The evaluation sampling strategy is summarized in Table 13-2. Cadmus evaluated all components with

basic levels of rigor. For all of the sampled components, Cadmus used simple random sampling to select

records for reviews. For the Audit and Kit stratum, Cadmus surveyed all eligible participants for the

online survey. Within the Efficient Equipment stratum, there were no duct sealing projects in PY10, so

Cadmus did not conduct a review of this measure. Cadmus will evaluate this measure in PY11. Cadmus

also did not sample Online Marketplace participants in PY10, because project data were not available.

Cadmus completed fewer records reviews than planned within the equipment component due to small

population sizes. The evaluation of the pilot will occur in PY11. The gross impact evaluation activities

produced results with ±2.82% precision at 85% confidence.

Table 13-2. PY10 Energy Efficient Home Program Gross Impact Sample Design

Stratum Population

Size


Design

Target Sample Size



New Homes 1,002 0.5 40 40 REM/Rate modeling review

Audit and Kit (1) 7,852

0.5 80 80 Records review (2)

0.5 All eligible

participants 234 Verification online survey (3)

Weatherization 765 0.5 60 40 Records review (4)

Efficient Equipment

10,244 (5) 0.5 160 124 Records review (6)

Program Total 19,863 (7) 518

(1) Includes online assessments and in-home audits; both channels delivered energy-savings kits to customers. (2) Cadmus sampled 40 in-home audit projects and 40 online assessment projects. (3) Online survey conducted with online assessment participants. (4) Cadmus sampled 20 insulation projects and 20 air sealing projects. There were no duct sealing projects in PY10.

(5) The number of unique rebates available in PPL Electric Utilities’ tracking database. (6) Cadmus conducted records reviews for 31 water heater fuel switching, 23 ASHP tune-up, 30 dehumidifiers, and 40 ductless heat pumps. (7) Program total population size is lower than reported population in Table 13-1 because three ductless heat pump records (outdoor units) had no reported savings and no associated indoor units. Cadmus removed these participants from the count of verified participants.




Table 13-3. Energy Efficient Home Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 9,943 18,802 17,661 46,407 (1)


In PY10, the Energy Efficient Home Program reported energy savings of 20,434 MWh/yr, as shown in

Table 13-4, and demand reduction of 3.59 MW/yr, as shown in Table 13-5.

Table 13-4. Energy Efficient Home Program Gross Impact Results for Energy



Error Ratio


C.L. PYVTD MWh/yr

New Homes 3,696 72% 0.13 2.92% 2,672

Audit and Kit (1) 2,967 66% N/A 6.30% 1,944

Weatherization 977 80% 0.62 18.37% 781

Efficient Equipment 12,794 96% 0.44 4.85% 12,264

Program Total (2) 20,434 86% N/A 3.56% 17,661

(1) Includes online assessments and in-home audits; both channels delivered energy-savings kits to customers. (2) Program total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

Table 13-5. Energy Efficient Home Program Gross Impact Results for Demand



Error Ratio


C.L. PYVTD MW/yr

New Homes 0.93 72% 0.53 11.88% 0.67

Audit and Kit (1) 0.30 66% N/A 5.84% 0.20

Weatherization 0.09 83% 0.42 12.70% 0.07

Efficient Equipment 2.28 97% 0.12 1.06% 2.20

Program Total (2) 3.59 87% N/A 2.68% 3.14

(1) Includes online assessments and in-home audits; both channels delivered energy-savings kits to customers. (2) Program total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

A number of factors led to variation between the reported and verified savings and to the observed

realization rates of less than or greater than 100% for energy savings and/or demand reduction.

Additional information can be found in Appendix K for each of these program components:

• New Homes. Cadmus found that most of the variation between reported ex ante and verified ex

post savings was caused by the lighting and appliance assumptions the ICSP used to calculate

ex ante energy and demand savings, which was similar to evaluation findings in PY9. Cadmus


evaluated the energy savings reported in the REM/Rate models provided by the ICSP’s

subcontractor and determined that the ICSP used REM/Rate assumptions to calculate all

savings, which is inconsistent with the PA TRM. The REM/Rate software overestimates lighting

energy savings. The software calculates energy savings for every appliance even if the appliance

was not installed. These factors led to a realization rate of 72% for energy and demand.

• Audit and Kit. For the Audit and Kit stratum, Cadmus conducted a records review for the in-

home audit and online assessment offerings and collected data to calculate installation rates

(ISRs) via the online participant survey. The Audit and Kit stratum had a 66% realization rate for

energy and demand.

▪ In-home audits had a 47% energy realization rate and a 48% demand realization rate. Online

assessments had a 66% energy realization rate and a 67% demand realization rate. The main

issue contributing to the low realization rates was low ISRs for kit products; Cadmus found

ISRs to be lower than those used by the ICSP in its reported savings.81

▪ Another factor affecting the in-home audit realization rate was in-home audit

documentation showing that the auditor did not setback the water heater temperature;

however, the temperature (post-setback) was filled in for these fields in the tracking data.

Cadmus applied zero savings in these instances.

• Weatherization. The Weatherization stratum had an 80% energy realization rate and an 83%

demand realization rate. Within this stratum, insulation had a 79% energy realization rate and

an 82% demand realization rate; air sealing had a 100% realization rate for energy and demand.

▪ For insulation projects, the major inconsistency was due to data entry errors. Cadmus

verified that all PPL Electric Utilities tracking database inputs were correct but the

realization rates were below 100% for 98 projects.

▪ Other errors in insulation projects included missing installation dates for the central air

conditioning systems, incorrect previous R-values, incorrect existing inches of insulation,

and incorrect added R-values.

• Efficient Equipment. Overall, the Efficient Equipment stratum had a 96% energy realization rate

and a 97% demand realization rate. Cadmus’ database and records review found various errors

across ductless heat pump, central heating equipment fuel switching, water heater fuel

switching, heat pump water heater, smart thermostat, and ENERGY STAR refrigerator and

ENERGY STAR dehumidifier participants. Additional information can be found in Appendix K.1.4

Gross Impact Evaluation: Efficient Equipment.

81 The evaluated PY10 ISRs for in-home audit and online assessment kit products can be found in Appendix K.1.5 New Homes REM/Rate Modeling Findings.


▪ The majority of the errors were among dehumidifier projects. The ICSP used current

ENERGY STAR standards for dehumidifiers, 82 but Cadmus followed the PA TRM which uses

the 2012 ENERGY STAR standard.83 Cadmus discussed the same issue in the PY9 Annual

Report.84 In addition, a few records’ model numbers did not meet either the 3.0 or 4.0

ENERGY STAR specifications for dehumidifiers. Cadmus applied zero ex post savings to these

records. The realization rate for dehumidifiers was 63% for energy and demand.

▪ For ductless heat pumps, Cadmus found several issues with the AHRI numbers and the

indoor and outdoor unit model numbers. The realization rates for ductless heat pumps were

96% for energy and 97% for demand.


In PY10, Cadmus used the results of PY8 and PY9 participant surveys to calculate free ridership and

spillover and determine net savings for all Energy Efficient Home Program offerings. Cadmus attempted

to collect survey data to inform net savings calculations for the ASHP tune-up, fossil fuel switching water

heater, and duct sealing participants, but no participants completed surveys for these new offerings in

PY10. A detailed explanation of the methodology used to calculate net savings and the findings from PY9

and PY8 can be found in Appendix J.2 of the PY8 and PY9 Annual Reports.85,86



Table 13-6 shows the Efficient Equipment stratum free ridership, spillover, and NTG ratios by equipment

category.

82 EPA. ENERGY STAR Program Requirements for Dehumidifiers Version 4.0. October 25, 2016. Accessed online: https://www.energystar.gov/products/appliances/dehumidifiers/key_efficiency_criteria.

83 See section 2.4.8 of the PA TRM. This section uses ENERGY STAR Version 3.0.

84 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to PA PUC. Prepared by Cadmus. November 15, 2018. Available online: http://www.puc.pa.gov/pcdocs/1595564.pdf

85 Ibid.


https://www.energystar.gov/products/appliances/dehumidifiers/key_efficiency_criteria




Table 13-6. Energy Efficient Home Program – Efficient Equipment Stratum Net Impact Evaluation Results

Equipment Category PYVTD kWh/yr

Evaluation Year

Free Ridership

(%) (1)

Spillover (%)

NTG Ratio

Refrigerator 104,125 PY9 63% 11% 0.48

Dehumidifier 319,227 PY9 48% 16% 0.68

HVAC 8,737,050 PY8 44% 7% 0.63

HPWH 865,686 PY8 22% 7% 0.85

Other 2,237,655 PY8 47% 7% 0.60

Stratum Total (2) (3) 12,263,743 43% 7% 0.64

(1) These estimates were weighted by the survey sample-verified program kWh/yr savings. This method ensured that respondents who achieved higher energy savings through the program had a greater influence on the equipment-level free ridership estimate than did respondents who achieved lower energy savings. (2) Equipment-level free ridership, spillover, and NTG Ratio estimates were weighted by the product’s verified kWh/yr program population savings to arrive at the Efficient Equipment stratum NTG ratio of 0.64. (3) May not match due to rounding.

Table 13-7 shows the free ridership, spillover, and NTG ratios by program component.

Table 13-7. Energy Efficient Home Program Net Impact Evaluation Results

Stratum PYVTD kWh/yr

Evaluation Year

Free Ridership

(%) (1)

Spillover (%)

NTG Ratio

New Homes 2,672,428 PY8 51% 0% 0.49

Online Assessment Kit 1,908,536 PY8 7% 9% 1.02

Audit – In-Home 38,881 PY8 4% 1% 0.97

Weatherization 780,809 PY8 49% 7% 0.58

Efficient Equipment 12,263,793 PY9(2) & PY8 43% 7% 0.64

Program Total (3) (4) 17,664,447 40% 6% 0.66

(1) These estimates were weighted by the survey sample-verified program kWh/yr savings. This method ensured that respondents who achieved higher energy savings through the program products had a greater influence on the equipment-level free ridership estimate than did the respondents who achieved lower energy savings. (2) Refrigerators and dehumidifiers were the only products evaluated for NTG ratio in PY9. PY8 NTG ratio results were used for all other equipment categories of the Efficient Equipment stratum. (3) The stratum-level free ridership, spillover, and NTG ratio estimates were weighted by the product’s verified kWh/yr program population savings to arrive at the final Energy Efficient Home Program NTG ratio of 0.66. (4) Total may not match sum of rows due to rounding.




demand savings estimates to calculate the verified savings estimates for the Energy Efficient Home

Program in PY10.

Table 13-8. PYTD and P3TD Energy Efficient Home Program Savings Summary




PYVTD Net (2) 11,593 1.98

P3RTD Gross 52,760 9.22

P3VTD Gross 46,407 8.47

P3VTD Net (2) 32,477 5.59

(1) Total may not match sum of rows due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy savings compliance target.



The main research objective for the Energy Efficient Home PY10 process evaluation was to assess

customer satisfaction.


The PY10 process evaluation for the Energy Efficient Home Program included these activities:



• Online participant surveys

These activities were consistent with the evaluation plan, except for changing the schedule of the Online

Marketplace evaluation. Cadmus and PPL Electric Utilities agreed to move the Online Marketplace

evaluation to PY11 because no participant data were uploaded to PPL Electric Utilities’ tracking database

in PY10. Table 13-9 shows the sampling strategy for the process evaluation.


Table 13-9. Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size (1)

Assumed Proportion


Target Sample

Size






Program Stakeholders

PPL Electric Utilities Program

and ICSP Staff


2 N/A 2 2 N/A N/A

Participants (1)

In-Home Audit and Kit

Online survey

376 90/10 All

records 6 101 100%

Online Assessment

Online survey

6,244 90/10 All

records 234 5,796 100%

Equipment Online survey

3,391 90/10 All

records 210 2,357 100%

Weatherization Online survey

1,639 85/15 All

records 38 517 100%

Program Total 11,652 490 8,771

(1) For participants, population refers to unique projects at the time of the survey. (2) Achieved sample size is based on number of respondents answering the Overall Satisfaction question. Cadmus tracks this as a completed survey to estimate confidence and precision around satisfaction metrics. Some respondents completed surveys but did not answer the Overall Satisfaction question. Therefore, data captured from additional surveys contributed to various analyses discussed in this report. The number of responses is indicated in the discussion. (3) Sample frame is a list of participants with contact information who have a chance to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities’ tracking database at the time of the survey. After selecting all unique records, Cadmus removed any records from the population if the customers had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (email or telephone number), were on the national Do Not Call list, or opted out of the online survey. (4) Percent contacted means the percentage of the sample frame contacted to complete surveys.


Participants completed 488 online surveys, as shown in Table 13-9. Cadmus contacted all eligible

respondents for each program component and administered the online survey in waves throughout

PY10. The survey was administered during Q1 and Q3 for the equipment and online assessments, and

Q2 and Q4 for the in-home audits and weatherization projects to provide timely respondent feedback

and information to PPL Electric Utilities and the ICSP. The sample sizes reported throughout this report

may vary by survey question as respondents could skip questions if they chose not to answer; therefore,

not all respondents provided answers to every question. Cadmus included all survey respondents who

answered the satisfaction question when calculating overall satisfaction even if they did not complete

the entire survey.

Additional details about the approach to contacting customers and the sample attrition are presented in

Appendix K.2.2 Survey Approach and Appendix R Survey Methodology.


In February and March 2019, Cadmus conducted interviews with program staff from PPL Electric Utilities

(n=1) and the ICSP (n=1). The interviews focused on identifying and assessing how the program design


and delivery changed from PY9 to PY10, what worked well, where there may be challenges, and where

the program is headed in PY11.


The following sections describe the program delivery and satisfaction findings. Additional detail

regarding Cadmus’ approach to assessing response differences by survey mode and survey attrition are

in Appendix K.



Participant satisfaction with the Efficient Equipment, Audit and Kit (online assessments and in-home

audits), and Weatherization program components are reported in this section.87

In PY10, 87% of respondents (n=495) reported they were satisfied with the Energy Efficient Home

Program (±2.5% at 90% confidence).88 Sixty-one percent were very satisfied and 26% were somewhat

satisfied (Figure 13-1).Overall satisfaction (combining very satisfied and satisfied responses) in PY10 did

not significantly differ from that of respondents in PY8 or PY9, but the percentage of respondents

reporting that they were very satisfied is significantly lower in PY10 than in PY9.89 Additionally, 6% of

respondents reported dissatisfaction with the program (combining not too satisfied and not at all

satisfied responses) in PY10. This is a statistically significant increase relative to PY9 when 3% of

respondents reported dissatisfaction with the program.90

There are some differences in program satisfaction among participant groups in PY10. This is consistent

with respondent satisfaction from PY9:91

• Respondents in the equipment component reported 93% very satisfied or somewhat satisfied

(n=208)

• Respondents in the Weatherization component reported 90% very satisfied or somewhat

satisfied (n=38)

• Respondents in the online assessment component reported 81% very satisfied or somewhat

satisfied (n=243), which is significantly lower than other program components.

87 No interviews were conducted for the New Homes component in PY10.

88 Cadmus included all survey respondents who answered the satisfaction question even if they did not complete the entire survey.

89 Cadmus tested significance at the 90% confidence interval using a two-tailed t-test (p-value = 0.0533).

90 p-value = 0.0108

91 The sample of in-home audit participants (n=6) is too small to evaluate program satisfaction individually.


Figure 13-1. Overall Program Satisfaction for PY10, PY9, and PY8

Source: Question, “Now, thinking about your overall experience with PPL Electric Utilities'

(weatherization rebate/online home energy assessment/in-home audit/efficient equipment rebate) program,


Boxes around data labels indicate statistical significance at the 90% confidence interval using a two-tailed t-test. Cadmus

included all participants who answered the satisfaction question even if they did not complete the entire survey.

Participants were also asked about their satisfaction with various elements of program delivery (Figure

13-2). When considered across all participant groups in the aggregate, respondent satisfaction did not

change significantly from PY9 to PY10 for any of the three delivery elements. However, when groups of

participants are disaggregated, as shown in Table 13-10, Cadmus found that respondents from the

online home energy assessment component were significantly less satisfied with the clarity of

application requirements and the rebates for qualifying equipment and services compared to

respondents from the equipment component. This finding is consistent with PY9 and suggests that the

online energy assessment program component may be a focus area for program improvement. Full

satisfaction results by program component are in Appendix J, Section K.2.1. Additional Findings


Figure 13-2. PY10 Program Feature Satisfaction: All Program Delivery Elements

Source: Questions, “Please indicate how satisfied you are with each of the following program components:

clarity of application requirements, information you have learned online from PPL Electric Utilities about how to save energy,

and PPL Electric Utilities’ rebates for qualifying energy-efficient equipment and services.”

Table 13-10. PY10 Satisfaction: Program Delivery Elements by Component (Percent Satisfied or Very Satisfied)

Area Assessed Equipment

(A)

Weatherization

(B)

Online Assessment

(C) Total

Clarity of application

requirements

87%C

(n=206)

76%

(n=34)

74%A

(n=200)

80%

N=446

Rebates for qualifying

equipment and services

95%C

(n=207)

89%C

(n=38)

58%AB

(n=165)

80%

(n=416)

Information about how to save

energy

86%

(n=194)

84%

(n=32)

81%

(n=219)

83%

(n=451)

Note: Lettering and a highlighted cell denotes statistical significance between programs at the 90% confidence interval.

Areas Working Well

When asked which areas of the program were working well, respondents (n=488) most frequently

selected the rebate or energy-savings kit.92 Figure 13-3 depicts the areas working well in the PY10

Energy Efficient Home Program across the Efficient Equipment, Weatherization, and Audit and Kit (in-

home audit) components. Figure 13-4 depicts the areas working well in the PY10 Audit and Kit (online

assessment) component.

Participants in the Efficient Equipment, Weatherization, and Audit and Kit (in-home audit) components

were significantly more likely to say the time it took to receive the rebate or the energy-savings kit was

92 This was a closed-ended question.


working well (37%) compared to online assessment participants (22%).93 Conversely, online assessment

participants were significantly more likely to say the PPL Electric Utilities website (30%) was working well

compared to Efficient Equipment, Weatherization, and in-home audit participants (21%).

Figure 13-3. Areas Working Well: Equipment, Weatherization, and In-Home Audit Components

Source: Question, "Thinking about what worked well with the PPL Electric Utilities (weatherization rebate/in-home

audit/efficient equipment rebate) program, what one item worked best? What worked next best?" (Equipment n=210,

Weatherization n=38, In-Home Audit n=6) Multiple responses allowed.

Figure 13-4. Areas Working Well: Online Assessment Component

Source: Question, "Thinking about what worked well with the PPL Electric Utilities online home energy assessment program,


93 Significance tested at the 90% confidence interval using a two-tailed t-test (p-value = 0.0004).




calculated the TRC benefits using gross verified impacts. The net present value program year to date

(NPV PYTD) benefits and costs are expressed in PY10 dollars (PY10 includes months in both 2018 and

2019). Net present value costs and benefits for P3TD financials are expressed in PY8 dollars.

Table 13-11. Summary of Energy Efficient Home Program Finances–Gross Verified






EDC CSP EDC CSP



7 Marketing (4) - $275 - $625






$6,168 $6,275


$22,007 $46,686



16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits $122 $501



19 TRC Benefit-Cost Ratio (9) 0.78 0.73 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7 )Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.



Table 13-12. Summary of Energy Efficient Home Program Finances–Net Verified






EDC CSP EDC CSP



7 Marketing (4) - $275 - $625






$3,948 $3,472


$14,865 $32,802



16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits $124 $501





Cadmus quantified non-energy benefits in accordance with the SWE’s Guidance Memo.94 A summary of the methodologies Cadmus used to calculate the non-energy benefits of natural gas savings is presented in Appendix P Non-Energy Benefits.




Overall, the Energy Efficient Home Program was successfully delivered and administered in PY10. The program achieved 17,661 MWh/yr in

savings, which was 123% of the PY10 projected savings. The majority of participants, 81%, were very or somewhat satisfied with the program

offerings in which they participated. The ICSPs are working to correct realization rates for New Homes with software vendors (section K.1.3 New

Homes REM/Rate Modeling Findings) so Cadmus has no new findings for the New Homes component in PY10. Cadmus identified a few small

changes that the ICSP can make to improve the program’s realization rate moving forward.


Conclusion 1: The online

assessment participants are not as

satisfied as participants in the

Weatherization and Efficient

Equipment program components,

and, they would like more ways to

save energy.

Findings Support

• While the majority were satisfied, participants in the online assessment component were significantly less

satisfied with the program (81% very satisfied or somewhat satisfied, n=243) than respondents in the equipment

or weatherization components. (See section 13.5.3.1 Program Satisfaction and Appendix K.2.1 Additional

Findings.)

• When asked what they would like changed about the program (open-ended), participants in the online

assessment component most often requested more or other items in the kit and help understanding how to use

these items (n=27 of 63) and help understanding opportunities for energy savings beyond the items provided in

the kit (n=14 of 63). This feedback suggests that the current kit is a factor contributing to the difference in

satisfaction between online assessment participants and participants in weatherization and equipment program

components. The feedback also indicates a strong desire among participants to realize additional energy savings

(n=14 of 63). (See section 13.5.3.1 Program Satisfaction and Appendix K.2.1 Additional Findings.)

Conclusion 2: Improving the clarity

of the rebate application process

may increase satisfaction with this

element of the program.

Findings Support

• Eighty percent of participants were satisfied with the clarity of application requirements (n=446). (See section

13.5.3.1 Program Satisfaction).

• The top aspect of the program that participants think could be improved is the application process, reported by

27 participants. (See section 13.5.3.1 Program Satisfaction.)


Conclusion 3: The ICSP used

different inputs than the TRM to

calculate reported savings for

dehumidifiers.

Findings Support

• The dehumidifier component had energy and demand realization rates of 63%. (See section 13.2.2 Gross

Impact Evaluation Results.)

• The main reason for the low realization rate was that the ICSP used the 2016 ENERGY STAR-qualified unit

liters of water per kWh/yr consumed values (L/kWhee) for the capacity of the efficient unit rather than the

values listed in the PA TRM, which are the 2012 ENERGY STAR standard values. There were also instances

where the model number did not meet either the 3.0 or 4.0 ENERGY STAR specifications for dehumidifiers

(Cadmus applied zero ex post savings to these instances). (See section 13.2.2 Gross Impact Evaluation

Results.)

Conclusion 4: The ICSP incorrectly

calculated insulation savings;

addressing this will improve the

realization rate.

Findings Support

• The insulation component had an energy realization rate of 79% and a demand realization rate of 82%. (See

section 13.2.2 Gross Impact Evaluation Results).

• The major inconsistency in insulation records appears to be data entry errors such as a missing installation

date for the central air conditioning system, incorrect previous R-value, incorrect existing inches of

insulation, and incorrect added R-value. (See section 13.2.2 Gross Impact Evaluation Results.)

Conclusion 5: In-home auditors

are not setting back the water

heater temperature as reported.

This and evaluated ISRs are

leading to a low realization rate

for the Audit and Kit stratum.

Findings Support

• The online assessment component had an energy realization rate of 66% and a demand realization rate of

67%. (See section 13.2.2 Gross Impact Evaluation Results.)

• The in-home audit component had an energy realization rate of 47% and a demand realization rate of 48%.

• Cadmus found that the in-home auditors did not setback the water heater temperature, but the

temperature (post-setback) was entered in the tracking data. (See section 13.2.2 Gross Impact Evaluation

Results.)

• Cadmus used evaluated ISRs for kit products delivered through the in-home audit and online assessment

components. These were lower than ISRs used in reported savings calculations, affecting the realization

rates for the stratum. (See section 13.2.2 Gross Impact Evaluation Results and Appendix K.1.5 Installation

Verification Methodology and Findings.)


Table 13-13. Status of Recommendations for the Energy Efficient Home Program

Energy Efficient Home Program


EDC Status of Recommendation

(Implemented, Being Considered, Rejected

and Explanation of Action Taken by EDC)

Conclusion 1: The online assessment participants

are not as satisfied as participants in the

Weatherization and Efficient Equipment program

components, and, they would like more ways to

save energy.

Consider offering two to three pre-made kits with different products so

that participants can ”pick a kit” (from one of these pre-made kits),

designed to their needs and interests.

Being considered.

Consider providing additional instructional elements to help

participants better understand how to install the items in the kit. For

example, add how-to videos on the PPL Electric Utilities website.

Being considered.

Consider adding a link at the end of the online assessment for

customers to schedule an in-home audit. Being considered.

Consider adding a link at the end of the online assessment back to

“Ways to Save” and the Online Marketplace. Being considered.

Conclusion 2: Improving the clarity of the rebate

application process may increase satisfaction with

this element of the program.

As recommended in PY9, consider adding information to the PPL

Electric Utilities website to assist customers in understanding the

rebate form. If possible, add additional instructions to the rebate form

itself.

Being considered.

Conclusion 3: The ICSP used different inputs than

the TRM to calculate reported savings for

dehumidifiers.

Cadmus suggests the ICSP use the current PA TRM to calculate ex ante

savings, even though more recent ENERGY STAR values are available. Implemented.

Conclusion 4: The ICSP incorrectly calculated

insulation savings; addressing this will improve the

realization rate.

Cadmus suggests the ICSP use the PA TRM defaults and assign an R-

value of 5 where there are no existing inches of insulation. Where

previous insulation exists, the ICSP could use a linear extrapolation

using the PA TRM defaults to report the existing R-value.

Implemented.

Conclusion 5: In-home auditors are not setting

back the water heater temperature as reported.

This and evaluated ISRs are leading to a low

realization rate for the Audit and Kit stratum.

The ICSP could consider adjusting their calculations for reported

savings for kit products to include evaluated ISRs from PY10. Being considered.

Consider investigating why auditors are not setting back water heater

temperatures. If setbacks are not performed, the field on the

application form listing water heater temperature should be left blank

(indicating no setback was completed).

Being considered.

The program offers an incentive to customers who turn in eligible appliances and provides freepick-up and environmentally sound recycling services.

APPLIANCE RECYCLING PROGRAM


PY10 expenditures

$6,097

$2,078

93%

89%

of projected

of projected


PY10 saved

Phase III has so far saved 33,938 MWh/yr

11,362 MWh/yr

52%

92%

of projected

of projected



91%

93%

97%


satisfied with the clarity of application requirements

satisfied with PPL Electric Utilities rebates for qualifyingequipment and services

satisfied with the information learned online from PPL Electric Utilities

2,185

A total of 14,210 participants recycled:

Dehumidifiers

Room air conditioners

Freezers

9,060 Refrigerators

1,949

1,016

PY10 PARTICIPATION

Chapter 14 Appliance Recycling Program PPL Electric Utilities | 193

14 Appliance Recycling Program

In the Appliance Recycling Program, PPL Electric Utilities offers an incentive to customers who turn in

eligible appliances and provides free pick-up and environmentally sound recycling services. Refrigerators

must measure between 10 and 30 cubic feet to qualify for the program. Both primary and secondary

refrigerators and freezers are eligible. Eligible appliances must be plugged in and functioning when

picked up. If customers recycle a refrigerator or freezer, they can also turn in room air conditioners and

dehumidifiers. Though these appliances are not typically picked up as a stand-alone service, in PY10 PPL

Electric Utilities did offer two separate bulk recycling events where only these were eligible.

Table 14-1 shows the appliance eligibility parameters and incentives.

Table 14-1. Eligible Equipment and Incentives for the Appliance Recycling Program

Equipment Eligibility Rating Incentive Range

Refrigerator Working unit; > 10 cubic feet and ≤ 30 cubic feet Between $20 and $75

Freezer Working unit; > 10 cubic feet and ≤ 30 cubic feet Between $20 and $75

Room Air Conditioner Working unit removed from mounting Between $10 and $25

Dehumidifiers Working unit Between $10 and $25

PPL Electric Utilities’ energy efficiency programs staff provides overall strategic direction and program

management. Its EM&V staff oversees evaluation activities and coordinates with the program’s delivery

staff.

In PY10, CLEAResult, the ICSP, delivered the Appliance Recycling Program to customers and was

responsible for marketing and managing call center services, online and telephone scheduling of

appliance pick-ups, processing applications and rebates, tracking program data, and providing customer

and transaction information to PPL Electric Utilities. Recleim, the ICSP’s subcontractor, managed the

pick-up, decommissioning, and recycling of appliances.


The objectives of the Appliance Recycling Program are these: 95

• Encourage customers to dispose of their

existing, inefficient refrigerators, freezers,

air-conditioning units, and dehumidifier

units in an environmentally responsible

manner

• Reduce the use of secondary, inefficient

refrigerators, freezers, and air-conditioning

units

• Decommission appliances on the site to

prevent resale in a secondary market

• Promote other PPL Electric Utilities’ energy

efficiency programs

• Achieve a total energy reduction of

approximately 65,000 MWh/yr gross verified

savings

• Achieve high customer satisfaction with the

program

• Enhance relationships with box stores and

independent retailers to encourage

participation in the “buy new and recycle”

component



Cadmus defined participants as unique appliances that were decommissioned through the Appliance

Recycling Program during the program year. The program is targeted primarily to residential customers

but is available to all PPL Electric Utilities customers with a working, residential-grade refrigerator,

freezer, room air conditioner, or dehumidifier.


Table 14-2 presents the participation counts, reported and verified energy and demand savings, and

incentive payments for the Appliance Recycling Program in PY10 by customer segment.

Table 14-2. PY10 Appliance Recycling Program Participation and Reported Impacts

Parameter GNE Large C&I

(Non-GNE) Residential

Small C&I (Non-GNE)

Total⁽¹⁾

PYTD Number Participants 90 13 13,980 127 14,210

PYRTD MWh/yr 101 11 14,047 137 14,295

PYRTD MW/yr 0.01 0.00 2.13 0.02 2.17

PYVTD MWh/yr 79 9 11,167 107 11,362

PYVTD MW/yr 0.01 0.00 1.81 0.02 1.84

PY10 Incentives ($1000) $1 $0 $403 $0 $404


95 Program objectives are stipulated in PPL Electric Utilities revised Energy Efficiency and Conservation Plan Act 129 Phase III, EE&C plan (Docket No. 2015-2515642), November 2018.




Cadmus calculated gross verified savings by gathering data from the PPL Electric Utilities tracking

database and from surveys of program participants and used this information as inputs to the savings

algorithms specified in the 2016 Pennsylvania Phase III Technical Reference Manual (PA TRM). The

impact evaluation sampling strategy is listed in Table 14-3.

Table 14-3. PY10 Appliance Recycling Program Gross Impact Evaluation Sample Design

Stratum Population

Size




Appliance Recycling 14,210 N/A (1) 14,210 Database review

Program Total 14,210 N/A 14,210

(1) Because this program’s evaluation did not include sampling, Cv and target precision are not meaningful.



Table 14-4. Appliance Recycling Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 11,844 10,731 11,362 33,938 (1)


Table 14-5 shows the Appliance Recycling Program reported energy savings of 14,295 MWh/yr for PY10.

Table 14-5. PY10 Appliance Recycling Program Gross Impact Results for Energy




PYVTD MWh/yr (1)

Appliance Recycling 14,295 79% 0.06 10.04% 11,362

Program Total (2) 14,295 79% N/A 10.04% 11,362

⁽¹⁾ Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) Total may not match sum of rows due to rounding. Program realization rate excludes unverified savings.


Table 14-6 shows the number of verified units recycled in PY10 and the verified energy savings by product.

Table 14-6. PY10 Gross Energy Results by Product Recycled

Product PYVTD MWh/yr Product Count

Refrigerator 8,702 9,060

Freezer 1,547 2,185

Room air conditioner 264 1,949

Dehumidifiers 849 1,016

Program Total (1) 11,362 14,210


Table 14-7 shows a reported demand reduction of 2.17 MW/yr in PY10.

Table 14-7. PY10 Appliance Recycling Program Gross Impact Results for Demand




85% C.L.

PYVTD MW/yr (1)

Appliance Recycling 2.17 85% 0.06 10.04% 1.84

Program Total (2) 2.17 85% N/A 10.04% 1.84

⁽¹⁾Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2)Total may not match sum of rows due to rounding. Program realization rate excludes unverified savings.

As recommended by the Phase III Evaluation Framework,96 in PY10 Cadmus verified all of the open

variables, rather than relying on PA TRM default values. Cadmus used two data sources: PY10 tracking

data for physical appliance characteristics and PY10 survey data to determine primary or secondary

status and the rooms in which appliances were used prior to recycling.97 Additionally, Cadmus calculated

and applied verified part-use factors for refrigerators and freezers using PY10 participant survey

responses rather than applying the PA TRM default part-use values.

In PY10, PPL Electric Utilities reported gross savings per appliance using the default inputs for the

regression equation provided in the PA TRM, causing some differences in ex ante and ex post energy

savings.


97 Ibid. Section 2.3.4.2 Using the TRM to Determine Ex Post Savings.


The following factors led to variation between the reported and verified savings and to the observed

realization rate:98

• Average refrigerator age was seven years younger than the PA TRM default (22.8 years rather

than 29.4 years).

• Average freezer age was 10 years younger than the PA TRM default (27.7 years rather than 37.5

years).

• The proportion of recycled refrigerators used as the primary refrigerator was 54% rather than

the 65% default from the PA TRM.

• Refrigerator unit energy consumptions (UECs) were 11% lower with the verified open variable

inputs compared to the PA TRM default. Freezer UECs were 18% lower.

• For refrigerators, the verified part-use factor was 87.0% compared to the PA TRM default of

96.9%. For freezers, the verified part-use factor was 80.0% compared to the PA TRM default of

98.5%. That verified part-use factors were considerably lower than the PA TRM defaults is

somewhat expected, because the source for the PA TRM was last updated in PY3. More

information about Cadmus’ methodology for calculating part-use is contained in Appendix L.1.


In PY10, Cadmus calculated the NTG ratio using the methodology described in the Common Methods for

Appliance Recycling programs specified by the SWE (Phase III Evaluation Framework, Appendix B).99 This

is consistent with the Uniform Methods Project (UMP) appliance recycling protocol to determine

program net savings.100 The NTG ratio determined in PY10 is 0.66. For more information on net savings

methodology, see Appendix L.

Table 14-8 shows that historical and current NTG ratios, determined using primary data, remained stable

over the life of the program, between 0.60 and 0.70, with the exception of a temporary uptick in PY5.101

98 For a complete list of calculated variables used to populate the TRM regression equations, refer to Table L-4 in

Appendix L.1.2.


100 National Renewable Energy Laboratory. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures. “Chapter 7: Refrigerator Recycling Evaluation Protocol.” September 2017. https://www.nrel.gov/docs/fy17osti/68563.pdf


https://www.nrel.gov/docs/fy17osti/68563.pdf


PY7 and PY9 are not included in the table because Cadmus applied the PY6 and PY8 NTG ratios in those

years, respectively. 102,103,104,105

Table 14-8. Current and Historical Net-to-Gross Ratios

Program Year Net-to-Gross Ratio

PY10 0.66

PY8 0.66

PY6 0.60

PY5 0.74

PY4 0.68

PY3 0.63

PY2 0.61


Table 14-9 shows the reported energy savings (PYRTD) and the verified gross and net energy savings

estimates calculated by Cadmus for the Appliance Recycling Program in PY10.

Table 14-9. PYTD and P3TD Appliance Recycling Program Savings Summary




PYVTD Net (2) 7,499 1.21

P3RTD Gross 39,784 5.71

P3VTD Gross 33,938 5.05

P3VTD Net (2) 22,399 3.33

(1) May not match due to rounding. (2) Net savings are not used to meet PPL Electric Utilities’ energy savings compliance target.


103 PPL Electric Utilities. Annual Report Program Year 9: June 1, 2017–May 31, 2018. Presented to Pennsylvania Public Utility Commission. Prepared by Cadmus. November 15, 2018. Available

online: http://www.puc.pa.gov/pcdocs/1595564.pdf


105 PPL Electric Utilities. Annual Report Program Year 8: June 1, 2016–May 31, 2017. Presented to Pennsylvania Public Utility Commission. Prepared by Cadmus. November 15, 2017. Available

online: http://www.puc.pa.gov/pcdocs/1544671.pdf







The purpose of the PY10 process evaluation was to assess and provide recommendations for improving

the Appliance Recycling Program’s effectiveness by evaluating customer satisfaction with the program.


The PY10 process evaluation for the Appliance Recycling Program included these activities:



• Online participant survey

The PY10 evaluation activities were consistent with the evaluation plan. The nonparticipant surveys

were conducted to inform net savings and are discussed in Section 14.3 Net Impact Evaluation and in

Appendix L.2 Net Impact Evaluation. Table 10-8 lists the process evaluation sampling strategy.

Table 14-10. PY10 Appliance Recycling Program Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size


Records Selected

for Sample

Frame (2)



Program Staff and ICSP

PPL Electric Utilities, CLEAResult

Phone and Email

4 N/A 2 4 4 All

Participants (Q1 and Q3)

Appliance Recycling

Online survey

6,205 (4) - As many as possible

368 3,078 100%

(1) Number includes only completed surveys. Respondents could skip questions. (2) Sample frame is a list of participants with email contact information drawn from the PPL Electric Utilities’ database. After selecting all unique records, Cadmus removed any records from the population if the customers had participated in a survey in the last three months, were selected for another program survey, did not have valid contact information (email or telephone number), were on the do not call list, or opted out of the online survey. (3) Percent contacted means the percentage of the sample frame contacted to complete surveys.

(4) Number of rebates for refrigerators and freezers available in PPL Electric Utilities’ tracking database at the time of the Q1 and Q3 survey efforts.


Cadmus contacted all Appliance Recycling Program participants with email addresses who recycled

refrigerators and freezers in Q1 and Q3. Of these, 368 participants completed the online survey. The

survey produced a measurement of program satisfaction with ±2% precision at 90% confidence.

Additional details about Cadmus’ approach to contacting customers and the sample attrition are

presented in Appendix L.3.1 and Appendix R Survey Methodology.



In February 2019, Cadmus interviewed Appliance Recycling Program managers at PPL Electric Utilities

(n=2) and the ICSP (n=2). The interviews focused on identifying and assessing changes to program design

and delivery from PY9 to PY10, understanding areas that are working well and any possible challenges,

and following up on recommendations made in PY9.



The Appliance Recycling Program was delivered effectively in PY10. Figure 14-1 shows the percentages

of appliances recycled in PY10.

Figure 14-1. Proportion of Equipment Recycled in PY10 (Percentage of Total Units)

Source: PPL Electric Utilities Tracking Database (n=14,210 Q1-Q4 participants)

Program Marketing

In previous program years, the Appliance Recycling Program has been communicated clearly to

customers through bill inserts, which remained an important marketing channel in PY10. As in PY9, the

ICSP’s tracking data for PY10 confirmed that bill inserts from PPL Electric Utilities was the most common

way customers learned about the program.

Table 14-11 shows the ways participants learned about the program. Table 14-12 shows the most

common digital sources of program information. Cadmus derived these data from the ICSP tracking

database, which has the most detailed responses to this question and includes all program participants.


Table 14-11. How Participants Learned About the Appliance Recycling Program

How Customer Learned About Program

PY8 (n=11,125)

PY9 (n=12,852)

PY10 (n=14,210)

Bill Insert 49% 43% 43%

Word of Mouth 20% 21% 22%

Online/Digital Sources [1] 15% 19% 21%

In-Store/Salesperson 5% 5% 5%

Other 10% 12% 9%

Source: ICSP tracking data question, “How did you hear about the program?” The total may not add to 100% due to rounding. In the PY8 and PY9 annual reports, Cadmus provided these data by age group for the subset of participants whose age was collected in the participant survey, therefore the percentages are not directly comparable. [1] Details in Table 14-12.

Table 14-12. Digital Sources for How Participants Learned About the Appliance Recycling Program

How Customer Learned About Program

PY8 (n=1,723)

PY9 (n=2,399)

PY10 (n=2,949)

PPL Electric Utilities Website 61% 51% 56%

Internet Search 33% 30% 24%

Email 5% 17% 8%

Social Media 1% 1% 12%

Digital Banner 0% 0% 0%

Source: ICSP tracking data question, “How did you hear about the program?” The total may not add to 100% due to rounding. In the PY8 and PY9 annual reports, Cadmus provided these data by age group for the subset of participants whose age was collected in the participant survey, therefore the percentages are not directly comparable.


Overall, 87% of PY10 participants were very satisfied with the program and 10% were somewhat

satisfied (n=379). The PY10 results were consistent with previous years (Figure 14-2).


Figure 14-2. Participant Overall Satisfaction

Source: Participant survey question, “Now, thinking about your overall experience with

PPL Electric Utilities Appliance Recycling Program, how would you rate your satisfaction?”

Participants also showed high levels of satisfaction for individual program components (Figure 14-3).

Participants were asked about their satisfaction with the rebates, clarity of application requirements,

and online information about ways to save energy.

Figure 14-3. PY10 Participant Satisfaction With Program Components

Source: Participant survey question, “Please indicate how satisfied you are

with each of the following program components.”

Areas Working Well

When asked what aspects of the program worked well, 45% of respondents said the rebate amount and

43% said the application process (n=368), as shown in Figure 14-4.


Figure 14-4. Program Components That Worked Best or Next Best for Participants

Source: Participant survey question, “Thinking about what worked well with the program,

what one item worked best? What worked next best?” (n=368).

Percentages exceed 100% because multiple responses were allowed.



calculated the TRC benefits using gross verified impacts. The net present value program year to date

(NPV PYTD) benefits and costs are expressed in PY10 dollars (PY10 includes months in both 2018 and

2019). Net present value costs and benefits for P3TD financials are expressed in PY8 dollars.


Table 14-13. Summary of Appliance Recycling Program Finances–Gross Verified


1 EDC Incentives to Participants $404 $1,027



4 Incremental Measure Costs (Sum of rows 1 through 3) (1) $404 $1,027

EDC CSP EDC CSP



7 Marketing (4) - $232 - $550






- -

13 Total NPV TRC Costs (Net present value of sum of rows 4, 11, and 12) (7) (1)

$2,078 $5,666







(1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total.



Table 14-14. Summary of Appliance Recycling Program Finances–Net Verified


1 EDC Incentives to Participants $404 $1,027



4 Incremental Measure Costs (Sum of rows 1 through 3) (1) $404 $1,027

EDC CSP EDC EDC



7 Marketing (4) - $232 - $550




11 (6) Program Overhead Costs (Sum of rows 5 through 10) (1) $1,674 $4,640


- -


$2,078 $5,666









The Appliance Recycling Program has continued to run smoothly over the course of PY10. As in previous

years, customers reported high levels of satisfaction with the program. The program verified savings are

lower than the EE&C plan’s projected savings, because the PA TRM’s UEC equations use outdated inputs

to compute savings. However, the ICSP updated these inputs in PY11 and the realization rate should

improve, which will provide program planners a more accurate estimate of expected savings. Though

the updated UEC inputs will provide better realization rates, the program itself may not reach its

projected energy savings at the current level of customer engagement. Nevertheless, the program is still

a strong contributor to the residential sector, and the residential sector as a whole is on track to meet

the Phase III compliance target.

Therefore, Cadmus has no recommendations to change the approach and processes that PPL Electric

Utilities and the ICSP use in this program to manage and achieve savings.

The program delivers energy education and kits with energy-saving products to income-qualified customers at or below 150% of the federal poverty income guidelines.

ENERGY EFFICIENCY KITS AND EDUCATION PROGRAM


PY10 expenditures

$5,170

$1,244

124%

83%

of projected

of projected



PY10 saved

30,352 MWh/yr

9,304 MWh/yr

81%

116%

of projected

of projected




91%agreed the program helped themunderstand how much energythe kit items could save

89% agreed the kits had what they neededto install the products

PY10 PARTICIPATION

gency4,109

9,455 irect mail


Chapter 15 Energy Efficiency Kits and Education Program PPL Electric Utilities | 208

15 Energy Efficiency Kits and Education Program

Through the Energy Efficiency Kits and Education Program, PPL Electric Utilities delivers energy

education and kits with energy-saving products to income-qualified customers at or below 150% of the

federal poverty income guidelines. The program is available to customers in single-family housing and in

multifamily housing where each unit is metered (not master-metered).

The program uses two channels to recruit participants and deliver the program:

• Direct mail kits. CMC Energy, the ICSP, conducted targeted email and mail outreach to invite

qualified customers to participate in the Energy Efficiency Kits and Education Program. To

receive a kit in the mail, recipients had to sign up online with the seven-digit code included on

their mailer or with their account number. Otherwise, they could return the business reply card

attached to the mailing. To generate the list of targeted outreach recipients, PPL Electric Utilities

identified customers who had received Low-Income Home Energy Assistance Program (LIHEAP)

benefits, were enrolled in PPL Electric Utilities’ OnTrack Program, or were low-income-qualified

and had been identified by the ICSP and PPL Electric Utilities through market research, data

mining, or other means.

• Agency delivery. Through their day-to-day interactions with clients, agencies (community-based

organizations, or CBOs) helped the ICSP’s subcontractor (Resource Action Programs, or RAP)

recruit qualified customers to participate in a one-hour energy education workshop or a one-on-

one session with agency staff at the agency’s office. RAP conducted train-the-trainer webinar

sessions for agency staff who requested training to explain the key elements of the workshops

and provide them with the tools they needed to introduce energy education and low-cost/no-

cost energy efficiency products to their low-income clients.

In PY10, the program distributed two kits—a base load kit and an electric kit—depending on the

customer’s fuel source for water heating because PPL Electric Utilities can claim savings only for water-

saving products installed in homes with an electric water heater.

Both kits contained self-installed products, energy education literature, and surveys to gather

participation information for the program. Kits for customers with electric water heaters also included

faucet aerators and low-flow showerheads. Kits for customers with a water heater fuel type other than

electricity did not contain aerators or showerheads.

Each kit also included a paper survey, along with a self-addressed, stamped envelope. Cadmus used the

survey-collected data to determine ISRs and satisfaction with the program. Table 15-1 lists the items

included in each kit.


Table 15-1. Products Included in PY10 Energy Efficiency Kits

Energy Efficiency Product Non-Electric Water

Heater Kit Electric Water

Heater Kit

Six 9W LED Bulbs ✓ ✓

Two LED Night Lights ✓ ✓

One Furnace Whistle ✓ ✓

Two Low-Flow Showerheads ✓

One Kitchen Aerator ✓

Tips on Energy Efficiency Behavior ✓ ✓

Paper Survey ✓ ✓

The objectives of the Energy Efficiency Kits and Education Program are these:106

• Provide low-income customers with a no-cost energy efficiency kit and education to help them

conserve energy and reduce their energy costs

• Maintain partnerships with local agencies so customers receive maximum and timely customer

assistance

• Achieve high satisfaction with customers and participating agencies, through quality service and

an impactful program offering

• Promote other PPL Electric Utilities low-income energy efficiency and assistance programs

• Achieve a total approximate reduction in energy use of 38,000 MWh/year gross verified savings



A participant in the program is defined as an income-eligible customer who received an energy-savings

kit through the agency or the direct-mail delivery channel. For recordkeeping purposes, each kit is

assigned a unique job number. Customers who receive more than one kit are assigned multiple job

numbers, one per unique kit.

Any kits returned to the ICSP receive two unique job numbers: one to indicate the distributed kit, and

one to indicate the returned kit. For recordkeeping purposes, returned kits appear as separate records

with negative reported savings in PPL Electric Utilities’ tracking database.


Table 15-2 presents the number of records in PPL Electric Utilities’ tracking database, the participation

counts (distributed kits that were not returned) and reported energy and demand savings for the Energy

Efficiency Kits and Education Program by customer segment in PY10. There are no incentive payments



for this program. Income-qualified customers receive the kit for free. See Appendix M.1.2 Database

Review Findings for details about the count of kits.

Table 15-2. Energy Efficiency Kits and Education Program Participation and Reported Impacts

Parameter Low-Income Total⁽¹⁾

PYTD # Participants (2) 13,932 13,932

PYTD Number of Participants Receiving Kits (3) 13,564 13,564

PYRTD MWh/yr 12,083 12,083


PYVTD MWh/yr 9,304 9,304


PY10 Incentives ($1000) $0 $0

⁽¹⁾ Total may not match sum of columns due to rounding. (2) The number of records is determined by the unique job numbers. Returned kits are assigned two unique job numbers:

one for the distributed kit, and one for the returned kit. Note that this is just for recordkeeping purposes. (3) In PY10, the ICSP distributed 13,756 unique kits. A total of 192 kits were returned, represented as 368 unique rows in PPL

Electric Utilities’ tracking database. In some cases, a kit distributed in PY9 was returned in PY10, and thus only had one

record in the tracking database.



Cadmus completed the following activities to evaluate the gross impacts of the Energy Efficiency Kits

and Education Program. Refer to Appendix M.1 Gross Impact Evaluation for detail on these activities.

• Records review. Cadmus reviewed the records in PPL Electric Utilities’ tracking database and

compared these to the records in the enrollment data provided by the ICSP, discussing

discrepancies with the ICSP prior to conducting any analyses.

• Participant kit survey review. Cadmus collected the kit surveys returned by mail to the ICSP and

used the collected data in the ex post savings analysis. Cadmus also reviewed the records in the

survey data and verified all discrepancies between the survey records and PPL Electric Utilities’

tracking database with the ICSP.


Cadmus collected data to verify energy savings through the ICSP-administered participant surveys

(paper surveys included in each kit). Cadmus also collected enrollment information from the ICSP’s

subcontractor to confirm the records in PPL Electric Utilities’ tracking database.


The paper survey included in each energy-savings kit asked questions about installing the products and

about the participant’s experiences with the products and program. Participants returned the surveys to


the ICSP throughout the year. When the program year ended in May 2019, the ICSP sent the survey data

to Cadmus. Cadmus used the data to estimate the program’s energy savings in PY10.

The impact evaluation sampling strategy is shown in Table 15-3. Additional details about methodology

are in Appendix M.1 Gross Impact Evaluation.

Table 15-3. Energy Efficiency Kits and Education Gross Impact Sample Design for PY10

Stratum Population

Size (1)

Assumed

Proportion or Cv

in Sample Design

Target

Sample

Size

Achieved

Sample

Size (2)


Agency 4,109 N/A (3) N/A (3) 195 ICSP-collected paper kit survey

Direct Mail 9,455 N/A (3) N/A (3) 753 ICSP-collected paper kit survey

Program Total 13,564 N/A (3) N/A (3) 948

(1) Number of unique kits that were distributed and not returned to the ICSP (this is not the number of unique records in PPL

Electric Utilities’ tracking database, which includes returned kits). (See Appendix M.1.2 Database Review Findings for details

about the count of kits.) (2) Number includes partially completed surveys. Respondents could skip questions. (3) Cadmus used survey responses collected by the ICSP from all participants who returned their surveys. Therefore, Cadmus

did not have an assumed proportion of Cv.



Table 15-4 Energy Efficiency Kits and Education Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 9,219 11,829 9,304 30,352 (1)


In PY10, the Energy Efficiency Kits and Education Program reported energy savings of 12,083 MWh/yr, as

shown in Table 15-5, and demand reduction of 0.82 MW/yr, as shown in Table 15-6.

Table 15-5. Energy Efficiency Kits and Education Program Gross Impact Results for Energy


Realization Rate

Sample Cv or

Error Ratio

Relative

Precision at

85% C.L.

PYVTD MWh/yr

Agency 3,601 75% 0.47 4.87% 2,697

Direct Mail 8,482 78% 0.48 2.51% 6,606

Program Total (1) 12,083 77% N/A 2.27% 9,304 (1) Total may not match sum of rows due to rounding. Due to rounding, multiplying the PYRTD savings by the realization

rate will not accurately reflect the final verified savings.


Table 15-6. Energy Efficiency Kits and Education Program Gross Impact Results for Demand


Realization Rate

Sample Cv or

Error Ratio

Relative

Precision at

85% C.L.

PYVTD MW/yr

Agency 0.24 113% 0.67 6.88% 0.28

Direct Mail 0.58 117% 0.66 3.48% 0.67

Program Total (1) 0.82 116% N/A 3.18% 0.95 (1) Rows may not sum to program total due to rounding. Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings.

The following factors affected the program’s achievements in PY10:

• The ICSP switched to a ship-a-kit system in PY10 so that agency clients could receive their kits

directly in the mail instead of transporting the kit home from the agencies. This process change

led to an estimated 58% increase in kits distributed through agencies.

• The ICSP distributed almost 14,000 kits in PY10, many more than the 8,000 kits projected in the

EE&C Plan for PY10. PPL Electric Utilities and the ICSP made the decision to send more kits to

increase the savings achieved in the low-income sector prior to phasing out kits in PY12.

Differences in reported and evaluated ISRs primarily drove the differences in reported and verified

savings for most products (shown in Table 15-7 in the next section):

• Differences in reported and evaluated ISRs drove the differences in reported and verified

savings for furnace whistles, low-flow showerheads, and LEDs. See section 15.2.2.1 In-Service

Rates.

• For energy education, the ICSP reported savings of 253 kWh/yr for every participant, regardless

of stratum and kit type. Cadmus, however, found that customers who received kits with water-

saving products more frequently had electric water heating, electric space heating, and central

cooling systems and therefore had higher energy education savings than customers who did not

receive water-saving products. Cadmus estimated energy education savings for recipients of

water-savings kits as 286 kWh/yr for agency and 343 kWh/yr for direct mail, similar to the ICSP’s

reported savings of 253 kWh/yr. This similarity is because of the high correlation between

having electric water heating (required to receive water-saving products) and having electric

space heating (required to achieve a large portion of the energy education savings).

However, Cadmus estimated far less energy education savings for recipients of kits with no

water-savings products (14 kWh/yr for agency and 51 kWh/yr for direct mail participants). The

ICSP’s assumption for per-kit savings for energy education does not apply as well to kits with no

water-saving products, and therefore the composition of kit types distributed to customers each

year is the primary driver of the realization rates.

• The ICSP reported 0 kW/yr for energy education, and Cadmus found 0.0182 kW/yr on average

across strata, which increased the overall demand realization rates to 113% for the agency

stratum and 117% for the direct mail stratum.

See Appendix M.1 Gross Impact Evaluation for additional details.


15.2.2.1 In-Service Rates

Table 15-7 shows reported and evaluated ISRs by product and strata for PY10. Overall, reported ISRs are

reasonable and matched well to the ISRs Cadmus estimated using survey data. The reported ISRs for

furnace whistles continue to be low compared to the evaluated ISRs. On the other hand, LED bulbs

continue to achieve lower evaluated ISRs than reported, likely because of the additional bulbs included

in the Phase III kits. Cadmus observed that LED ISRs remained relatively high until after the fourth bulb,

when installations dropped off dramatically, ranging from 68% to 56% for the fifth and sixth bulbs.

Similarly, nightlights and low-flow showerheads achieved lower evaluated ISRs than reported because

the PY10 kits included an additional nightlight and showerhead. Installation rates for nightlights account

for the negative savings when a participant installed a new nightlight, instead of replacing an old

nightlight. Participants installed the first nightlight at a rate of 53% and 62% (agency and direct mail

participants, respectively), compared to 35% and 42% for the second nightlight. Participants installed

the first showerhead at a rate of 82% and 90% (agency and direct mail participants, respectively),

compared to 7% and 16% for the second showerhead. Table 15-7 shows the ISR for each product

category. See Appendix M.1.3 Survey Findings.

Table 15-7. PY10 Reported vs. Evaluated ISRs

Product Agency Direct Mail

Reported ISR Evaluated ISR Reported ISR Evaluated ISR

Furnace Whistle 17% 30% 20% 35%

LED Bulbs 96% 78% 98% 83%

Nightlight 87% 44% 92% 52%

Kitchen Faucet Aerator 63% 60% 75% 68%

Low-Flow Showerhead 64% 44% 72% 53%


Energy Efficiency Kits and Education Program is offered to income-eligible customers in the low-income

community. No free riders are anticipated among the population receiving the energy-savings kits

because income-constrained customers are not likely to purchase the items in these kits on their own.

An NTG ratio of 1.0 is appropriate for this program; therefore, Cadmus did not estimate net savings.


In Table 15-8, Cadmus applied the realization rates to the reported energy and demand savings

estimates to calculate the verified savings estimates for the Energy Efficiency Kits and Education

Program in PY10. Because the NTG ratio is 1.0, net savings are the same as verified gross savings.


Table 15-8. PYTD and P3TD Energy Efficiency Kits and Education Program Savings Summary




PYVTD Net(2) (3) 9,304 0.95

P3RTD Gross 34,708 2.47

P3VTD Gross 30,352 2.93

P3VTD Net (2) (3) 30,352 2.93

(1) Total may not match due to rounding.

⁽2⁾ Net savings are not used to meet PPL Electric Utilities’ energy savings compliance target. (3) Net savings are equal to gross savings because the program is assumed to have an NTG ratio of 1.0.



Cadmus conducted the PY10 process evaluation with a focus on program delivery and participation and

addressed the following research objectives:

• Identify areas of program successes and challenges

• Identify areas that may benefit from program improvements

• Assess satisfaction of customers receiving the kits


The PY10 process activities included the following:


• Interviews with community-based organizations (CBOs)

• Analysis of returned participant kit surveys

• Review of logic model

The research activities were consistent with the evaluation plan. Cadmus reviewed, but did not update

the logic model since the program is being delivered as expected and is on track to meet the anticipated

intermediate and long-term outcomes.



Table 15-9. PY10 Energy Efficiency Kits and Education Program Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size


Design

Target Sample

Size

Achieved Sample

Size


for Sample Frame


Achieve Sample (1)



3 N/A 3 3 N/A 100%

Participants(2)

Agency responders

Paper kit survey

4,109(3) N/A N/A 195 N/A N/A

Direct mail responders

Paper kit survey

9,455(3) N/A N/A 753 N/A N/A

Community-Based Organizations (CBOs)


18 N/A 5 5 N/A 27%

Program Total

N/A N/A 13,585 N/A N/A 956 N/A N/A

(1) Percent contacted means the percentage of the sample frame contacted to complete surveys. (2) Cadmus analyzed all surveys that were returned. (3) Number of unique kits verified and not returned to the ICSP, not unique jobs. See Table 15-2, Table 15-3, and Appendix M.1.2 Database Review Findings for more details.


Cadmus collected participant data from the paper survey included in the kits and returned to the ICSP.

Completed participant survey responses produced a measurement of program satisfaction with ± 0.61%

precision at 90% confidence. Of 13,564 kits distributed (and not sent back or returned to the agency),

948 participants returned the paper survey, a response rate of 7%. This response rate is lower than the

10% achieved in PY9.107 Cadmus used the participant responses and determined that energy and

demand ex post verified savings estimates exceeded 90% confidence and 10% precision (90/10) by

stratum (agency and direct mail) and for the program overall.

15.5.2.2 Program Staff, ICSP, and CBO Interviews

In May of 2019, Cadmus conducted three interviews with the Energy Efficiency Kits and Education

Program manager from PPL Electric Utilities and the two ICSPs. The interviews focused on any changes

made to the program from PY10 and on program successes and challenges.

Cadmus also conducted five interviews with representatives of CBOs that deliver the program to their

clients. The interviews focused on agency satisfaction with the program, program successes, and

program challenges. Randomly selected agencies participated in the interviews, including two low-




distributing agencies, two high-distributing agencies, and one middle-distributing agency. Cadmus

categorized agencies as “low,” “medium,” or “high” distributors based on the number of kits each

distributed, with the bottom third categorized as “low” and the top third categorized as “high.”



Overall, Cadmus found that the ICSP continued to deliver the Energy Efficiency Kits and Education

Program well in PY10. Both PPL Electric Utilities and the ICSP reported that the program was delivered

effectively through the two delivery channels (direct mail and 18 agencies).

Cadmus last conducted interviews with agency staff in the PY8 evaluation and found that many agencies

struggled to distribute kits to customers for two reasons. Agencies had limited space to store kits at

their locations, and some customers had difficulty transporting the kit home. In PY10, these challengers

were resolved because PPL Electric Utilities and the ICSP switched to a “ship-a-kit” system, through

which customers could have the kit mailed to their homes after the agency helped them enroll in the

program via phone or an in-person meeting.

In PY9, the ICSP reported receiving a lot of questions from agencies and program participants about the

advanced power strip, specifically because this item has a more complex installation process than other

products in the kit. PPL Electric Utilities chose to remove the advanced power strip from the kit for PY10

in order to find savings in a more cost-effective way.

Program Changes

In PY10, program stakeholders made five primary changes to the program:

• Kit contents. In both the base load kit and the electric kit, PPL Electric Utilities added a night

light and removed the advanced power strip. It added an extra showerhead to the electric kit

only.

• Kit contact information. PPL Electric Utilities added the phone number for RAP’s customer call

center to the pamphlets included in the kits, with the intent of increasing program engagement

and ISR by providing customer assistance. In previous program years, PPL Electric Utilities

printed the phone number of RAP’s customer call center on the box and not on any materials

inside the kit.

• Kit delivery. PPL Electric Utilities and the ICSP adopted a ship-a-kit system. Participating

agencies arrange to have the ICSP mail kits directly to customers after enrolling them in the

program. Previously, agencies distributed kits from their physical locations, which required

agencies to have on-site storage and meant customers had to carry the kits home. Agencies still

provide education to customers about the kit and receive payment from PPL Electric Utilities but

no longer have to store adequate supplies of kits on site.


• Program marketing approach. PPL Electric Utilities and the ICSP marketed the program to

customers via email for the first time in PY10. After determining which customers met the

program’s low-income criteria, the ICSP sent emails to eligible customers who had an email

registered with PPL Electric Utilities. If the customer had not enrolled in the program, the ICSP

mailed a postcard to these households a week later.

• Enrollment process. Customers who received email marketing could enroll in the program

online using their account number instead of the ID number included in the direct mailer.

PPL Electric Utilities reported that the program will end after PY11 because of new lighting standards

from EISA in 2020 (EISA 2020), at which time LEDs will not provide as much energy savings. Kit

distributions are expected to cease sometime in December 2019 after the program meets its Phase III

plans for participation and energy savings.

Marketing and Outreach

Marketing and outreach methods in PY10 described by the ICSP, PPL Electric Utilities, and agencies

remained consistent with previous program years. Two agencies said they marketed the kits to every

client who received agency services, but the other three said they marketed the program only to

customers who participated in related assistance programs. Agencies reported using the following

marketing and outreach methods:

• Calls and visits to other area nonprofits or partner agencies (four agencies)

• Marketing the program to clients who participate in other assistance programs (i.e., OnTrack,

budgeting workshops, and weatherization workshops) (five agencies)

• Fliers in the agency building (two agencies)

Cadmus analyzed the returned paper survey included in each participant’s kit. The survey asked

participants how they learned about the program. Results were mostly consistent with PY9. Direct mail

participants most frequently reported learning about the program through a direct mailer (64%, n=745),

while agency participants most frequently reported learning about the program through an agency

(34%, n=193). However, significantly fewer agency participants reported learning about the program

through an agency than were reported in PY9 (64%; n=166).108 This is likely a result of the ship-a-kit

program, where agency participants now enroll through an agency but receive their kit in the mail,

causing confusion in some agency participants about how they initially learned about the program.

Energy Education

All agencies reported providing some form of energy education to program participants when delivering

the kits. Each agency provided one-on-one education in person, and three of the five agencies also

provided education during workshops about budgeting or weatherization.

108 Cadmus used a two-tailed t-test with Bonferroni correction to test for statistical significance. The p-value is 0.0001.




As shown in Figure 15-1, customers continue to be satisfied with the energy efficiency kit they received

as part of the program in PY10; 97% of customers said they were either very satisfied (83%) or

somewhat satisfied (14%). Cadmus found no evidence that satisfaction differed between PY9 and

PY10.109

Most respondents (91%) agreed the program helped them understand how much energy the kit items

could save, and 89% agreed that the kits had what they needed to install the products.

Figure 15-1. Participant Satisfaction with Energy Efficiency Kit

Source: Survey question, “Now, thinking about your overall experience with the Energy Efficiency Kit,


Agency Satisfaction

Cadmus asked agencies about their overall satisfaction with the program. Four of five agencies

interviewed were very satisfied and one was somewhat satisfied with the program in PY10, consistent

with the satisfaction ratings from PY8. When asked about their satisfaction with the contents of the kit,

all were somewhat satisfied. Four agencies said that instead of removing the advanced power strip from

kits entirely in PY10, they would have liked for PPL Electric Utilities to include the less complicated

version of the advanced power strip that was in the PY9 kit.

Areas Working Well

Overall, Cadmus found that many aspects of the program worked well in PY10. Both direct mail and

agency delivery channels efficiently distributed kits to eligible customers. Compared to PY9, the program

109 Cadmus used a chi-square test for independence to determine whether responses in PY10 differed significantly from PY9. The p-value is 0.3867, indicating that there is no statistical difference in satisfaction between PY9 and PY10.


distributed slightly fewer kits through the direct mail channel (9,389 in PY10 versus 10,684 in PY9). Kits

distributed through the agency channel increased by 58% (3,989 in PY10 versus 2,519 in PY9).

All interviewed program and agency staff said the ship-a-kit system was a major program success. They

said the system streamlined the delivery process for agency staff as agencies no longer had to store kits

on site and customers no longer had to carry the kits home. One agency also said staff no longer had to

bring kits to off-site workshops. Another said the system made it easier for rural customers to receive

kits, since many do not want to travel to the agency to pick up a kit in person. One agency said the

system substantially increased the number of kits it could provide to clients.

Additionally, the ICSP said allowing customers to sign up for the program with their utility account

number instead of the seven-digit code on the mailers removed the sign-up barrier for customers who

may have misplaced their mailer.

PPL Electric Utilities said replacing the advanced power strip with a second showerhead was a PY10

program success. The second showerhead reduced the acquisition cost of energy savings for the

program. The change increased the cost-effectiveness per kit because the showerhead costs less than

the advanced power strip.

Challenges

PPL Electric Utilities, ICSP, and agencies reported a few areas that were not working well in PY10:

• PPL Electric Utilities, the ICSP, and its subcontractor agreed that email marketing was less

effective than anticipated at gaining new program participants. PPL Electric Utilities explained

that emailing is much less expensive than sending mailers, but only 1% to 2% of customers who

received an email enrolled in the program compared to 20% who received mailers.

• PPL Electric Utilities said the second showerhead reduced program realization rates for

showerheads compared to prior program years because most participating homes have only

one shower. The second showerhead’s lower acquisition cost for energy savings was a tradeoff

for the lower realization rates with second-showerhead installations.

• Only three of the five agencies interviewed had demonstration kits at their locations now that

the program uses a ship-a-kit system. Of the two that did not have a demonstration kit on site,

one said that its main office had one but its satellite offices did not. Both agencies said that,

without a demonstration kit, they had difficulty explaining its products and motivating their

clients to enroll in the program. PPL Electric Utilities is following up with agencies to make sure

they have a demo kit available.

Suggested Program Improvements

The interviewed agencies suggested improvements to the PY11 program, such as removing the

participation restriction for customers who recently participated in the PPL Electric Utilities

weatherization program. Some agencies suggested these updates to the kit contents:

• Add plastic window covers to prevent winter drafts

• Include a less complicated advanced power strip


• Remove the furnace filter whistle

• Provide a higher quality faucet aerator

• Add items to the kit that would generate customer excitement, such as motion-sensor lighting


Details of program finances and cost-effectiveness are presented in Table 15-10. Cadmus calculated TRC

benefits using gross verified impacts. The net present value program year-to-date (NPV PYTD) benefits

and costs are expressed in PY10 dollars (PY10 includes months in both 2018 and 2019). Net present

value costs and benefits for P3TD financials are expressed in PY8 dollars. Net verified savings are equal

to gross verified savings because the program is assumed to have an NTG ratio of 1.0.

In the SWE’s PY9 Annual Report,110 the SWE recommended treating the cost of kits as incentives to

participants in future program year reporting. Some difference of opinion exists as to what should be

called an incentive. The term can be interpreted broadly to include almost anything. Direct rebates,

interest payment subsidies, and even energy audits can be called incentives. Operationally, it is

necessary to restrict the term to include only dollar benefits such as rebates or rate incentives (monthly

bill credits). PPL Electric Utilities incorporates the cost of kits into the TRC as program delivery costs

rather than incentives to participants. PPL Electric Utilities considered changing its approach to conform

to the SWE’s request. However, since PPL Electric Utilities’ tracking and internal reporting systems are in

place to catalog these costs as a program delivery cost, it would be cost-prohibitive for PPL Electric

Utilities to change its processes and reporting procedures for Phase III. PPL Electric Utilities will consider

changing its approach in Phase IV, if required in the final TRC Order.


the methodologies Cadmus used to calculate the non-energy benefits of saved water, natural gas

therms, and lighting interactive effects can be found in Appendix P Non-Energy Benefits.

110 Pennsylvania Public Utility Commission. SWE Annual Report Act 129 Program Year 9. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, Brightline Group, and Demand Side Analytics, LLC February 28, 2019.

http://www.puc.pa.gov/Electric/pdf/Act129/Act129-SWE_AR_Y9_022819.pdf


http://www.puc.pa.gov/Electric/pdf/Act129/Act129-SWE_AR_Y9_022819.pdf


Table 15-10. Summary of Energy Efficiency Kits and Education Program Finances–Gross and Net Verified

Row

# Cost Category PYTD ($1,000) P3TD ($1,000) (10)





EDC CSP EDC CSP



7 Marketing (4) - $141 - $372








and 12) (7) $1,244 $4,856



16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits $421 $1,127




(1) May not sum to total due to rounding. (2) All costs for plan design and development are portfolio level costs and are assigned to customer sectors at the end of the phase.



legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs,





Overall, the Energy Efficiency Kits and Education Program has performed according to the program design and has exceeded both its projected

participation and projected savings. Recommendations are provided in Table 15-11, along with a summary of how PPL Electric Utilities plans to

address the recommendations.

Conclusion 1: The ship-a-kit system improved

program participation and streamlined program

delivery.

Findings Support

• Agency distribution of kits increased 58% in PY10 compared to PY9. (See section 15.2.2 Gross

Impact Evaluation Results.)

• All agency staff reported that that ship-a-kit system was a program success, reporting the

benefits of not needing to store kits onsite, expanding kit access to rural clients who do not visit

the agencies in person, and removing the need for clients to carry the kits home. (See section

15.5.3.2.)

Conclusion 2: Program realization rates depend on

the distribution of kit types in the program year

because of ex ante per-kit savings assumptions.

Findings Support

• Customers who received kits with water-saving products more frequently had electric water

heating, electric space heating, and central cooling systems and therefore had higher energy

education savings (286 kWh/yr for agency and 343 kWh/yr for direct mail) than customers who

did not receive water-saving products (14 kWh/yr for agency and 51 kWh/yr for direct mail

participants). (See section 15.2.2.)

• The ICSP’s assumption for per-kit savings for energy education (253 kWh/yr) does not apply well

to kits with no water-saving products. (See section 15.2.2.)

Conclusion 3: The low ISR for the second

showerhead warrants replacing the second

showerhead with another product.

• The ISR for the second low-flow showerhead was 7% in PY10 as most participants’ homes only

have one shower. (See section 15.2.2.1.)

• The addition of a second low-flow showerhead in PY10 did not help the program maintain savings

after the removal of the Tier II Smart Strips included in PY9 kits. (See section 15.2.2.1.)

• Agency staff recommended adding items to the kit that would generate customer excitement.

(See section 15.5.3.2.)


Table 15-11. Status of Recommendations Energy Efficient Kits and Education

Energy Efficient Kits and Education





Conclusion 1: The ship-a-kit system improved program

participation and streamlined program delivery. Continue using the ship-a-kit system. Implemented.

Conclusion 2: Program realization rates depend on the

distribution of kit types in the program year because

of ex ante per-kit savings assumptions.

The ICSP could consider reporting different energy

education savings for participants who receive kits with no

water-savings products to better forecast program savings

and increase realization rates.

Rejected.

Conclusion 3: The low ISR for the second showerhead

warrants replacing the second showerhead with

another product.

In future kit programs, consider removing the second

showerhead from the kits. Being considered.

Consider researching products that could be added to

replace the second showerhead. Being considered.

The program provides THINK! ENERGY, a school-based energy efficiency education curriculum, through classroom presentations to students and classroom materials for teachers.

STUDENT ENERGY EFFICIENT EDUCATION PROGRAM


PY10 expenditures

$3,160

$1,153

94%

135%

of projected

of projected



PY10 saved

16,573 MWh/yr



92%of projected



90% Bright Kids (2nd – 3rd grades)

PY10 PARTICIPATION

81% Take Action (5th – 7th grades)

78%

81%

Innovation (9th – 12th grades)

Innovation Pilot (9th – 12th grades)

5,054 Bright Kids (2nd – 3rd grades)

13,546 Take Action (5th – 7th grades)

2,517

2,548

Innovation (9th – 12th grades)

Innovation Pilot (9th – 12th grades)


Chapter 16 Student Energy Efficient Education Program PPL Electric Utilities | 225

16 Student Energy Efficient Education Program

The Student Energy Efficient Education (SEEE) Program provides THINK! ENERGY, a school-based energy

efficiency education curriculum, through classroom presentations to students and classroom materials

for teachers. THINK! ENERGY is offered once during the school year, typically in the fall.

Students receive educational materials and a take-home energy-savings kit of low-cost products to

install at home. Each kit delivered to a student is counted as a program participant. The energy-savings

kits are tailored to each grade level participating in the program and contain items such as LED bulbs,

low-flow showerheads, faucet aerators, and smart power strips. Each kit includes a Home Energy

Worksheet (HEW) that asks questions to track kit product installation rates as well as participant

demographics and program satisfaction.

CLEAResult, PPL Electric Utilities’ residential ICSP, identified National Energy Foundation (NEF) as the

subcontractor to the ICSP. The ICSP undertakes a broad spectrum of responsibilities that includes

marketing to and recruiting potential schools and teachers, creating curricula correlated with

Pennsylvania academic standards, securing support of the program components by the Pennsylvania

Department of Education, and assembling and shipping the energy-savings kits. PPL Electric Utilities

collaborates with the ICSP on the program’s strategic direction while maintaining the overarching Act

129 administrative, program support, evaluation, and data management systems. The ICSP provides

oversight and direction to its subcontractor.

The objectives of the SEEE Program are these:112

• Expand and promote energy efficiency literacy through education outreach programs

• Provide energy efficiency education to students offered through school assemblies and

classroom curriculum

• Confirm that energy efficiency education correlates to the Pennsylvania Department of

Education’s academic standards

• Provide students with take-home kits of energy efficiency products that can be installed at home

• Provide teachers with energy efficiency information, lesson plans, activities, training, materials,

and support for classroom use

• Obtain participation of approximately 85,000 students through 2021 and achieve approximately

18,000 MWh/yr of gross verified savings

• Achieve high customer (students and teachers) satisfaction with the program





The SEEE Program provides energy-savings kits to students in three cohorts:

• Bright Kids (2nd – 3rd grades)

• Take Action (5th – 7th grades)

• Innovation (9th – 12th grades)

In PY10, the ICSP and the ICSP’s subcontractor continued the Innovation Pilot (9th – 12th grades), which

Cadmus evaluated as a separate cohort. The Innovation Pilot provided Tier 2 smart power strips in place

of Tier 1 smart strips for a subset of Innovation cohort classrooms.

Each energy-savings kit distributed is counted as a participant and is recorded in the ICSP’s database and

PPL Electric Utilities’ tracking database with a school, classroom, and teacher identifier. This identifier

represents one classroom and is recorded with the number of kits distributed in that specific classroom.

PPL Electric Utilities did not collect or record utility account numbers of classroom students who

received a kit.


Table 16-1 presents the participation counts and reported energy and demand savings for the SEEE

Program in PY10 by customer segment (residential). The program does not offer incentives; the kits are

offered free of charge.

In PY10, a portion of savings were attributed to the low-income sector, determined using the

Pennsylvania Department of Education data specifying the percentage of students receiving reduced

and free lunches. (See discussion in Section 16.4.1.) In PY11, PPL Electric Utilities will reallocate a portion

of the program’s energy savings, budget, and participants from this residential program to the low-

income sector, reporting this information under WRAP.

Table 16-1. PY10 Student Energy Efficient Education Participation and Reported Impacts

Parameter Residential Low-Income Total⁽¹⁾

PYTD # Participants 23,665 - 23,665

PYRTD MWh/yr 6,075 - 6,075

PYRTD MW/yr 0.59 - 0.59

PYVTD MWh/yr 2,876 3,134 6,011

PYVTD MW/yr 0.30 0.33 0.62

PY10 Incentives ($1,000) $0 $0





Cadmus conducted a limited impact evaluation in PY10 for the SEEE Program due to the limited program

design and target population changes between PY9 and PY10.

For the PY10 impact analysis of the Bright Kids, Take Action, and Innovation cohorts, Cadmus calculated

ex post savings using the evaluated PY9 average per-kit ex post savings for each cohort.113 Cadmus also

conducted a database review to ensure per-kit ex post savings were applied to the correct quantity of

kits distributed to each classroom in PY10.

For the Innovation Pilot cohort, which provided Tier 2 smart strips in place of the Tier 1 smart strips,

Cadmus fully evaluated savings using PY10 survey data gathered through paper and online home energy

worksheets (HEWs). Cadmus used the PY10 data because improvements to the HEW in PY10 provided

more accurate data for calculating in-service rates (ISRs) for Tier 2 smart strips. Cadmus used the PY10

HEW survey data to estimate savings for all energy savings products included in the kits (LED bulbs, Tier

2 smart strips, and showerheads), with the exception of the water heater setback. To calculate water

heater temperature reduction, Cadmus used the average midpoint of the ranges provided in the PY9

HEW because these data were more robust than the updated PY10 data.114


Table 16-2 summarizes the impact evaluation’s sampling strategy. The impact evaluation activities

produced energy and demand savings with ± 0.93% and 0.87% precision respectively, each with 85%

confidence.

113 For the PY10 Take Action cohort, Cadmus used the weighted average per-kit ex post savings from the PY9 Take Action and Take Action Pilot cohorts.

114 The ICSP updated the PY10 HEW to simplify the water heater temperature reduction question, asking only whether the temperature was reduced, instead of asking for the degrees of temperature reduction. The reported savings calculation uses the TRM default (reduction of 11 degrees) for those who confirm they reduced their water heater temperature. However, because Cadmus found a large discrepancy between the Average temperature reduction reported in PY9 (between 5.1 and 5.4 degrees depending on cohort) and the TRM default (11 degrees), Cadmus used the average reported temperature reduction from the PY9 survey to calculate PY10 energy savings for water heater setback.


Table 16-2. PY10 Student Energy Efficient Education Program Gross Impact Evaluation Sample Design

Stratum Population

Size


Design(1)


Impact Evaluation Data Source

Bright Kids 2nd – 3rd grades

5,054 N/A 3,796 PY9 paper and online HEWs

Take Action 5th – 7th grades

13,546 N/A 10,016 PY9 paper and online HEWs

Innovation 9th – 12th grades

2,517 N/A 2,646(2) PY9 paper and online HEWs

Innovation Pilot 9th – 12th grades

2,548 N/A 2,038 (PY10)

765 (PY9)

PY10 paper and online HEWs; PY9 paper and online HEWs (water-heater setback only)

Program Total 23,665 N/A(1) ≤18,496 (1) Because this program’s evaluation did not include sampling, Cv and planned precision are not meaningful. (2) Achieved sample size is greater than population size because this represents all HEWs returned by students in PY9, which had a greater number of Innovation participants than in PY10



Table 16-3. Student Energy Efficient Education Program Savings PY8 Verified PY9 Verified PY10 Verified Phase III Verified

MWh/yr 4,539 6,024 6,011 16,573 (1)


The program exceeded its projected savings due to substantially greater participation in PY10 (23,665

participants) than planned (13,047 participants). PPL Electric Utilities’ EE&C Plan proposed adding a low-

income Student Energy Efficient Education offering to its portfolio in PY10. Therefore, the ICSP targeted

schools with low-income students, increasing program participation in PY10.

In PY10, the SEEE Program reported energy savings of 6,075 MWh/yr, as shown in Table 16-4, and

demand reduction of 0.59 MW/yr, as shown in Table 16-5.

Table 16-4. PY10 Student Energy Efficient Education Program Gross Impact Results for Energy



Relative Precision

at 85% C.L.

PYVTD MWh/yr (1)

Bright Kids 531 100% 0.11 1.03% 530

Take Action 3,741 99% 0.21 1.33% 3,703

Innovation 754 97% 0.13 2.20% 733

Innovation Pilot 1,049 100% 0.09 1.99% 1,045

Program Total⁽2⁾ 6,075 99% N/A 0.93% 6,011

⁽¹⁾ Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2)Total may not match sum of rows due to rounding.


Table 16-5. PY10 Student Energy Efficient Education Gross Impact Results for Demand


Realization Rate

Sample Cv or

Error Ratio

Relative

Precision at 85%

C.L.

PYVTD

MW/yr (1)

Bright Kids 0.06 101% 0.10 1.02% 0.06

Take Action 0.36 108% 0.20 1.26% 0.39

Innovation 0.07 99% 0.10 1.82% 0.07

Innovation Pilot 0.10 101% 0.08 1.77% 0.10

Program Total⁽2⁾ 0.59 105% N/A 0.87% 0.62

⁽¹⁾ Due to rounding, multiplying the PYRTD savings by the realization rate will not accurately reflect the final verified savings. (2) Total may not match sum of rows due to rounding.

Reported savings aligned closely with verified savings in PY10. The following factors contributed to the

99% overall realization rate for the program:

• The ICSP used inputs from the PY9 evaluation for the Bright Kids, Take Action, and Innovation

cohorts, resulting in nearly 100% realization rates for all three cohorts. Slight differences still

exist in each cohort because of differences in methodology. The evaluation approach used

average per-kit savings, incorporating correlations in survey-gathered inputs between products,

whereas the ICSP directly multiplied average survey-gathered inputs to estimate savings for

each product, which yielded slightly different results. Any additional differences were due to

rounding of survey inputs.

• For the Innovation Pilot cohort, though the overall realization rate was 100%, Cadmus identified

some differences between reported and evaluated savings for showerheads, smart strips, and

water heater setbacks that are helpful to understand for future alignment.

▪ Reported savings for smart strips in PPL Electric Utilities’ tracking database did not account

for instances when smart strips were installed in entertainment centers. Smart strips

installed in entertainment centers generate more savings than when installed in unknown

locations, increasing realization rates.

▪ The ICSP used higher ISRs to calculate reported savings for showerheads than Cadmus found

after analyzing PY10 surveys, decreasing the realization rate.

▪ Cadmus used survey-gathered data from PY9 to determine the degrees by which

participants lowered their water heater temperature settings to calculate ex post savings for

the water heater temperature setback component of the kit. The survey analysis indicated

that participants lowered their water heater temperature setting by 5.1 degrees,

approximately half of the ICSP’s value of 11 degrees, decreasing realization rates.

16.2.2.1 Innovation Pilot In-Service Rates

Table 16-6 shows the verified ISRs for each of the items in the energy-savings kit for the Innovation Pilot

for PY9 and PY10. Consistent with prior years, ISRs were higher for electric products (LED bulbs and

smart strips) than for the water-saving products (showerheads).


The ISR for Tier 2 smart strips was higher in PY10 than in PY9. The PY9 ISR does not reflect the actual ISR

in PY9 for these participants; in PY9 Cadmus could verify installations only for smart strips used in

entertainment centers because of the phrasing of the survey question on the ICSP-administered HEW.

The ICSP updated the question in PY10, so the ISR more accurately reflects smart strip installations.

Table 16-6. Verified Student Energy Efficient Education ISRs for Products by Year

Kit Product PY9 PY10

LED (4 bulbs)(1),(2) 78% 77%(3)

Showerhead(4) 41% 35%

Power Strip 58% 75% (1) ISR reflects the average of four bulbs in PY9 and PY10 each. (2) For LED bulbs in PY7, PY8, and PY9, Cadmus based the ISR on the ISRs reported in the survey and an installation rate

“trajectory” to include savings for all program bulbs assumed to be installed over time. For these, it incorporated the

recommendations of the Uniform Methods Project (UMP). (National Renewable Energy Laboratory. Uniform Methods

Project. Chapter 21: Residential Lighting Evaluation Protocol. Prepared by Apex Analytics, LLC. November 2014. Available

online: http://www.nrel.gov/extranet/ump/pdfs/ump-res-lighting-clean.pdf.) The UMP uses the findings from the 2014

California Upstream and Residential Lighting Impact Evaluation, which suggested that bulb installation rates could be as high

as 97% within four years of purchase. Discounting the future savings back to the current program year reduces the ISR from

97%. The PY9 evaluation used a weighted average nominal discount rate of 8.14% for all electric distribution companies

(EDCs). (3) Individual Trajectory PY10 LED ISR for Innovation Pilot – LED1 83%, LED2 79%, LED3 74%, LED4 71% (ISRs calculated from

surveys without “trajectory” calculation were LED1 65%, LED2 56%, LED3 46%, LED4 39%).

(4) Cadmus calculated water product ISRs by dividing respondents who installed the product in an electric water heat home

by respondents who answered the question and have electric water heat.


The SEEE Program is a select offering to schools, and kits are provided free of charge to teachers, who in

turn provide the kits to their students. No free riders are anticipated among the population receiving the

energy-savings kit. That is, Cadmus does not expect teachers to voluntarily purchase and provide kits to

students in the absence of the program. Likewise, because the kits are sent home with children as part

of the school’s curriculum and households do not purchase the kit, Cadmus assumes there is no free

ridership. In addition, spillover is not measured.

The program is assumed to have a net-to-gross (NTG) ratio of 1.0.



demand savings estimates to calculate the verified savings estimates for the Student Energy Efficient

Education Program in PY10.

http://www.nrel.gov/extranet/ump/pdfs/ump-res-lighting-clean.pdf


Table 16-7. PYTD and P3TD Student Energy Efficiency Education Program Savings Summary

Savings Type Energy (MWh/yr)⁽¹⁾ Total Demand (MW/yr)⁽¹⁾

PYRTD 6,075 0.59


PYVTD Net⁽²⁾,⁽³⁾ 6,011 0.62

P3RTD 16,790 1.62

P3VTD Gross 16,573 1.73

P3VTD Net⁽²⁾,⁽³⁾ 16,573 1.73

⁽¹⁾ Total may not match sum of rows due to rounding.

⁽²⁾ Net savings are not used to meet PPL Electric Utilities’ energy saving compliance target.

⁽³⁾ Net savings are the same as verified savings.

16.4.1 Verified Savings Attributed to the Low-Income Sector

The Student Energy Efficient Education Program was offered to schools in PPL Electric Utilities’ service

territory that offer free lunches to children from households with income below 130% of the federal

poverty level (FPL), a more conservative percentage than the 150% FPL used as the income qualification

guideline for Act 129 low-income programs. Reduced-fee lunches are offered to students of families

with incomes below 185% of the FPL, which includes families with incomes between 130% and 150% of

the FPL.

The Pennsylvania Department of Education publishes the number and percentage of reduced-fee and

free lunches for each school and each grade within the school.115 Cadmus used these published data to

determine the percentage of low-income participants in the Student Energy Efficient Education

Program, assuming that the percentage of students enrolled in the school free-lunch program was

representative of the percentage within any particular grade-level classroom participating in the

program. These savings were assigned to the low-income sector. Additionally, Cadmus assumed that

half the students who qualify for reduced-fee lunches met the 150% FPL guideline and assigned savings

for these students to the low-income sector.

Therefore, Cadmus assigned program-verified savings of 6,011 MWh/yr to the residential and

low-income sectors as follows:

• 3,027,318 kWh/yr savings for all students receiving free lunches assigned to the low-income

sector (50% of program total)

• 106,966 kWh/yr savings for half the students receiving reduced-fee lunches assigned to the low-

income sector (2% of program total)

• 2,876,474 kWh/yr savings assigned to the residential sector (48% of program total)

115 Pennsylvania Department of Education. “National School Lunch Program Reports.” Accessed November 2019. https://www.education.pa.gov/Teachers%20-%20Administrators/Food-Nutrition/reports/Pages/National-School-Lunch-Program-Reports.aspx




The process evaluation of the Student Energy Efficient Education Program focused on assessing student

participant satisfaction with the program.


The PY10 process evaluation activities for the Student Energy Efficient Education Program were these:


• Analysis of satisfaction questions on student-returned HEWs

Given the limited process evaluation in PY10 and no substantial changes to the program delivery,

Cadmus determined that a review and update to the program’s logic model was not needed at this time.

The research activities were consistent with the evaluation plan for the Student Energy Efficient

Education Program.


Table 16-8. Student Energy Efficient Education Program Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population

Size

Assumed

Proportion or

Cv in Sample

Design

Target

Sample

Size

Achieved

Sample

Size

Number of

Records

Selected

for Sample

Frame

Percent of

Sample Frame

Contacted to

Achieve

Sample (1)

Program

Staff and

ICSP

PPL Electric

Utilities,

CLEAResult

staff

Telephone in-

depth

interview

2 N/A (2) 2 4 N/A 100%

Students

Bright Kids,

Take Action,

Innovation,

Innovation

Pilot

ICSP

subcontractor-

administered

paper and

online HEWs

23,665 N/A (2)

All

returned

surveys

17,239 All eligible 100%

Program

Total 23,667 N/A 2+ 17,243 N/A N/A

(1) Percent contacted means the percentage of the sample frame contacted to complete surveys. (2) Because this program’s evaluation did not include sampling, Cv and target precision are not meaningful.

16.5.2.1 Home Energy Worksheets (Participant Student Surveys)

Student participants completed HEWs, which were developed and administered by the ICSP’s

subcontractor, either online or on the paper forms included in the energy-savings kits. The number of

completed surveys produced a measurement of program satisfaction with ±0.48% precision at 90%

confidence.



In February 2019, Cadmus conducted interviews with Student Energy Efficient Education Program

managers from PPL Electric Utilities (n=2) and the ICSP (n=2). The interviews focused on identifying and

assessing changes to program design and delivery from PY9 to PY10.


This section summarizes findings about program delivery and student satisfaction.


Overall, program delivery in PY10 remained consistent with PY9. In PY10, the ICSP’s subcontractor rolled

out the “augmented reality” application (app) for smartphones and tablets, referred to as the Energy

Sidekick app, to all Take Action participants (the program had previously piloted the app in the spring of

PY9 to a subset of classrooms in the Take Action cohort). The intent of the app was to further engage

students and their parents with the kit’s products at home. Additionally, in line with PY9, the ICSP and

ICSP’s subcontractor continued the Innovation Pilot, which substituted Tier 2 smart strips in place of the

Tier 1 smart strips for a subset of 50 classrooms.

To further increase student engagement with the presentations, kit products, and classroom materials

and exercises, the ICSP’s subcontractor made the following enhancements in PY10:

• Reintroduced the student poster contest

• Added games to the program website for students and teachers

• Developed Kahoot! quizzes for Innovation teachers to use in their classrooms, which included

more challenging mathematical exercises, in line with feedback from Innovation cohort teachers

in PY9 that requested more engaging and challenging activities for these high school students

• Offered a Facebook group for Innovation teachers, which included articles and additional

curriculum

• Added installation videos with sound to the program website and included installation

instructions for kit products

16.5.3.2 Student Satisfaction

Seventy-three percent of participants completed HEWs, a slight increase from 71% in PY9. The figure

below summarizes the results of the student satisfaction by cohort. Eighty-three percent of students

said they were very satisfied (61%) or somewhat satisfied (22%) with the program overall (Figure 16-1).

In line with prior program year findings, the cohort most frequently very satisfied was Bright Kids (78%).

Least frequently very satisfied were the Innovation (49%) and Innovation Pilot (52%) cohorts.


Figure 16-1. Participant Satisfaction with Student Energy Efficient Education Program Overall by Cohort

Home Energy Worksheet Q8 (Bright Kids) and Q23 (Take Action, Innovation, and Innovation Pilot):

“Please rate your overall satisfaction with the Think! Energy program.”


A detailed breakdown of program finances and cost-effectiveness is presented in Table 16-9. The TRC

benefits were calculated using gross verified impacts. NPV PYTD benefits and costs are expressed in

PY10 dollars (PY10 includes months in both 2018 and 2019). NPV benefits and costs for P3TD financials

are expressed in PY8 dollars. Net verified savings are equal to gross verified savings because the

program is assumed to have an NTG ratio of 1.0.






Table 16-9. Summary of Student Energy Efficient Education Program Finances–Gross and Net Verified






EDC CSP EDC CSP



7 Marketing (4) - $170 - $436

8 Program Delivery (5) - $957 - $2,342




12 NPV of increases in costs of natural gas (or other fuels) for

fuel switching programs - -

13 Total NPV TRC Costs (Net present value of sum of rows 4,

11, and 12) (7) (1) $1,153 $2,924


15 Total NPV Lifetime Electric Capacity Benefits $231 $623

16 Total NPV Lifetime Operation and Maintenance (O&M)

Benefits $491 $1,275

17 Total NPV Lifetime Non-Electric Benefits (Fossil Fuel, Water) $5,005 (11) $4,390



(1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the

phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and

legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site

visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery”

costs. (6) Rows 1-11 are presented in nominal dollars. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply

costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas

valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) All program year (PYTD) expenditures and benefits are discounted to PY8 dollars for the Phase (P3TD) total. (11) Cadmus did not accurately account for per-unit therms water savings by the quantities in PY9. This was corrected for the

PY10 analysis.

Chapter 16 Student… Program PPL Electric Utilities | 236


Overall, the SEEE Program performed well in PY10, distributing more kits than projected and exceeding

the program’s planned savings. Furthermore, satisfaction with the program was high, with 83% of

students reporting they were very or somewhat satisfied with the program. Cadmus does not have any

recommendations for program improvement at this time.

The Demand Response Program is for commercial and industrial (C&I) customers and for government, nonprofit, and education (GNE) customers to voluntarily reduce electricity demand during Act 129 demand response events.

DEMAND RESPONSE

Phase III expenditures through PY10

PY10 expenditures

$5,136

$2,753


Phase III has so far saved an average of

PY10 saved an average of

116.6 MW

111.5 MW121%

of compliance target

VERIFIED DEMAND REDUCTION



67% satisfied with the ICSP

75% satisfied with the timing of event notifications

73% satisfied with the online enrollment process

58

A total of 6 events with 60unique participants:

facilities curtailed 116.2 MW in August 6, 2018 event

facilities curtailed 109.5 MW in July 3, 2018 event

facilities curtailed 105.9 MW in July 2, 2018 event58

58

PY10 PARTICIPATION

127%of compliance

target

85%of projected

85%of projected

50facilities curtailed 102.6 MW in September 5, 2018 event

facilities curtailed 114.6 MW in September 4, 2018 event

facilities curtailed 120.2 MW in August 28, 2018 event54

50

Chapter 17 Demand Response Program PPL Electric Utilities | 238

17 Demand Response Program

To comply with the Pennsylvania Public Utility Commission’s Act 129 Phase III demand response

compliance targets, PPL Electric Utilities’ Demand Response Program must reduce its system load by an

average of 92 MW (measured at the generator level) overall demand response events during the last

four years of Phase III (PY9–PY12).117 In addition, PPL Electric Utilities is required to achieve a minimum

of 85% of the 92 MW compliance target, or 78.2 MW, during each event.

During Phase III, PPL Electric Utilities is operating the Demand Response Program for commercial and

industrial (C&I) customers and for government, nonprofit, and education (GNE) customers.

Compliance targets for demand response programs were established at the generator level, which

means load reductions measured at the customer meter must be increased to reflect transmission and

distribution losses (line losses). The peak demand impact estimates presented in this report have been

adjusted for these line losses. PPL Electric Utilities uses the following line loss percentages and/or

multipliers by sector:

• Small C&I = [8.75% or 1.0875] • Large C&I = [4.2% or 1.0420]

PPL Electric Utilities manages the implementation conservation service provider (ICSP) and provides

overall strategic direction for the program.

CPower, the ICSP, enrolls and contracts with customers to reduce electricity demand during Act 129

demand response events.118 After the summer season, the ICSP makes performance-based payments to

participating customers.119

Demand response events were initiated in accordance with Act 129 Phase III Implementation Order,

which requires a four-hour event on the following day when at least one hour of the PJM

Interconnection regional transmission organization (RTO) day-ahead forecast exceeds 96% of its forecast

of summer peak demand. According to the order, there can be a maximum of six events per program

year, and in PY10 all six events were called by September 5, 2018.

In PY10, PPL Electric Utilities initiated six load curtailment events, including two pairs of back-to-back

events. Each event occurred on a non-holiday weekday between 2:00 p.m. and 6:00 p.m.

The ICSP notified participating customers between 10:30 a.m. and 11:15 a.m. on the day before each

event. Before the event started, customers confirmed their participation for specific event hours by

117 Program objectives are stipulated on PPL Electric Utilities’ revised EE&C Plan (Docket No. M-2015-2515642) filed with the Pennsylvania PUC in July 2018 and approved in November 2018.

118 CPower, the ICSP, contracted with four PPL Electric Utilities’ customer facilities through the demand response aggregators NRG and Direct Energy.

119 In PY10, 28 customers representing 64 facilities enrolled in PY10; however, four customers representing four facilities did not participate in any events.


logging into the ICSP’s online platform. Customers had the option of participating in all or a subset of

event hours. In PY10, among 60 participant facilities and across six events, there were 106 instances of a

facility participating for fewer than four hours of an event.



A participant in the Demand Response Program in PY10 is defined as a customer facility that

participated in at least one of PPL Electric Utilities’ Act 129 demand response events. The ICSP enrolled

64 customers in PY10. During PY10, a total of 24 customers with 60 participating facility sites

participated in at least one Act 129 demand response event.

A dual-enrolled participant is a facility that participated in PPL Electric Utilities’ Demand Response

Program and a PJM demand response program. In PY10, all PPL Electric Utilities demand response

program participants were dual-enrolled participants. Table 17-1 reports the number of these

participating facilities and the incentives paid.

Table 17-1. Dual-Enrolled Participants

Dual-Enrolled Customer Facilities

Act 129-Only Customer Facilities

Incentives Paid to Dual-Enrolled Customers

Incentives Paid to Act 129-Only Customers

60 0 $1,878,600 $0

Dual-enrolled customers were enrolled in PPL Electric Utilities' Act 129 Demand Response Program and PJM demand response programs in PY10.


Table 17-2 presents the participation counts, reported demand reduction, and incentive payments for

the Demand Response Program in PY10 by customer segment and Act 129 event. In PY10 (summer of

2018), the program reported demand savings of approximately 106 MW on July 2, 109 MW on July 3,

121 MW on August 6, 106 MW on August 28, 119 MW on September 4, and 107 MW on September 5.

Large C&I customers accounted for between 92% and 97% of the reported demand savings for these

events.


Table 17-2. PY10 Demand Response Program Participation and Reported Demand Reductions

Parameter Small C&I

(Non-GNE) Large C&I

(Non-GNE) GNE Total (1)

PYTD Number of Participants (2) 30 20 10 60

Event 1, July 2, 2018, Reported MW 0.5 102.0 3.8 106.3

Event 2, July 3, 2018, Reported MW 0.4 104.0 4.0 108.5

Event 3, August 6, 2018, Reported MW 1.2 114.4 5.5 121.1

Event 4, August 28, 2018, Reported MW 0.9 102.1 2.9 106.0

Event 5, September 4, 2018, Reported MW 2.1 115.4 1.7 119.1

Event 6, September 5, 2018, Reported MW 1.6 103.7 1.7 106.6

Total Average Reported MW 1.1 106.9 3.2 111.3

PY10 Incentives ($1000) (3) $20 $1,804 54 $1,879

Note: The load impacts reported in this table have been grossed up to reflect transmission and distribution losses. (1) Total may not equal total of row due to rounding. (2) Number of participants who participated in at least one event, not the number who enrolled in the program (64). (3) Refers to total savings across all events and all event hours



According to the Act 129 Phase III Implementation Order, a maximum of six events can be called per

program year.120 In PY10, six events were called, and the last event occurred on September 5, 2018.

The impact evaluation sampling strategy is shown in Table 17-3. Cadmus analyzed consumption data to

estimate Act 129 load impacts for the population of participants (that is, there was no sampling).

However, for three facilities, it was not possible to estimate event savings for one or two events because

the interval kWh meter readings during the event were estimated, not actual.121 The number and

composition of participants varied between events, because the ICSP called upon different sets of

customers for each event.

120 Phase III Final Implementation Order. From the Public Meeting of June 11, 2015. Pennsylvania Public Utility Commission. Docket No. M-2014-2424864. Available at http://www.puc.pa.gov/pcdocs/1367313.doc.

121 This affected one small C&I customer during two events and two small C&I customers, each for one event.


Table 17-3. PY10 Demand Response Program Gross Impact Sample Design

Stratum Event Population

Size



PYRTD MW Impact Evaluation

Activity

Small C&I

July 2, 2018 30 N/A (Census) 30 0.5

Analysis of individual

participating facility loads was

performed for each event hour.

July 3, 2018 30 N/A (Census) 30 0.4

August 6, 2018 30 N/A (Census) 29 1.3

August 28, 2018 30 N/A (Census) 29 1.0

September 4, 2018 30 N/A (Census) 29 2.1


Large C&I

July 2, 2018 19 N/A (Census) 19 102.0

July 3, 2018 20 N/A (Census) 20 104.0

August 6, 2018 18 N/A (Census) 18 114.5

August 28, 2018 18 N/A (Census) 18 102.1



GNE

July 2, 2018 9 N/A (Census) 9 3.8

July 3, 2018 8 N/A (Census) 8 4.0

August 6, 2018 10 N/A (Census) 10 5.5

August 28, 2018 6 N/A (Census) 6 2.9



Program Total (1)

July 2, 2018 58 N/A (Census) 58 106.3

July 3, 2018 58 N/A (Census) 58 108.5

August 6, 2018 58 N/A (Census) 57 121.3

August 28, 2018 54 N/A (Census) 53 106.0



The load impacts reported in this table have been grossed up to reflect transmission and distribution losses. This table incorporates minor corrections to the PYRTD MW reported in the PPL Electric Demand Response Annual Report to the PA PUC (July 15, 2019). The only Event Program Total affected was August 6, 2018 PYRTD, which increased from 121.1 MW to 121.3 MW. (1) Totals are rounded and may not sum due to rounding.

Cadmus evaluated each facility’s demand savings by comparing the facility’s metered demand during

event hours with an estimated baseline. The baseline was estimated using either regression analysis or a

day-matching method.122 For each participant, Cadmus analyzed interval consumption data to identify

the most accurate baseline calculation method. Additional details about the evaluation and baseline

selection methodology are in Appendix O.

122 Cadmus applied standard day-matching baseline calculation methods such as selecting the seven days of the previous 10 with highest average demand in accordance with SWE guidelines.



PPL Electric Utilities is on track to meet its Phase III Act 129 Demand Reduction compliance target

specified in the Implementation Order. Figure 17-1 shows the PY10 evaluation impact findings. In PY10,

verified peak load reductions were 111.5 MW (equal to the average demand reduction over the six

demand response events), a realization rate of 100.2% relative to the reported (ex ante) load reduction.

The P3TD verified peak load reductions were 116.6 MW (the average load reduction over PY9 and PY10

event hours), which exceeds the Phase III compliance target of 92 MW. In addition, PPL Electric Utilities

met its per-event compliance target of at least 78.2 MW (85% of the total compliance target) in each

demand response event. The verified average demand savings exceeded PPL Electric Utilities’ Act 129

compliance target for Phase III by 20 MW.

Figure 17-1. Gross Verified Savings in Comparison to Act 129 Targets

Note: These reported load impacts are based on Cadmus analysis of participant AMI consumption data and have been

grossed up to reflect transmission and distribution losses.

Table 17-4 shows PY10 Demand Response Program achievements by sector.


Table 17-4. PY10 Demand Response Program Gross Impact Results for Demand

Stratum Event PYRTD MW Demand

Realization Rate

PYVTD MW (1) Standard

Error

Relative Precision at 90% C.L. (2)

Small C&I

Event 1 0.5 371% 1.9 0.08 7%

Event 2 0.4 308% 1.4 0.08 10%

Event 3 1.3 146% 1.8 0.08 7%

Event 4 1.0 163% 1.6 0.08 8%

Event 5 2.1 92% 1.9 0.08 7%

Event 6 1.6 115% 1.8 0.08 7%

Large C&I

Event 1 102.0 95% 97.2 4.63 8%

Event 2 104.0 98% 101.8 4.61 7%

Event 3 114.5 94% 108.1 4.36 7%

Event 4 102.1 112% 114.5 4.51 6%

Event 5 115.4 96% 110.9 4.52 7%

Event 6 103.7 96% 99.2 4.50 7%

GNE

Event 1 3.8 179% 6.8 0.30 7%

Event 2 4.0 156% 6.3 0.29 8%

Event 3 5.5 114% 6.3 0.29 8%

Event 4 2.9 142% 4.1 0.28 11%

Event 5 1.7 108% 1.8 0.23 21%

Event 6 1.3 122% 1.6 0.20 15%

Event (3)

Event 1 106.3 100% 105.9 4.64 8%

Event 2 108.5 101% 109.5 4.62 7%

Event 3 121.3 96% 116.2 4.37 6%

Event 4 106.0 113% 120.2 4.52 6%

Event 5 119.1 96% 114.6 4.52 6%

Event 6 106.6 96% 102.6 4.51 7%

Average 111.3 100% 111.5 1.85 3%

Average Phase III DR Event Performance 116.6 1.46 2%

This table incorporates minor corrections to the PYRTD MW and demand realization rates reported in the PPL Electric Demand Response Annual Report to the PA PUC (July 15, 2019). The only Event Program Total affected was August 6, 2018 PYRTD, which increased from 121.1 MW to 121.3 MW. The Average PYRTD MW is unaffected. (1) Based on Cadmus’ analysis of participant AMI consumption data. MW were grossed up to reflect transmission and distribution losses. (2) Precision accounts for covariances of savings across hours of each event but not between events.

The following factors may have contributed to differences between the reported and verified savings

and the realization rates that deviated from 100%:

• Estimated interval consumption readings. Cadmus could not estimate demand savings for three

small C&I facilities during one or two events because the interval kWh readings for event hours

were estimated and not actual readings.123

123 The affected events (with number of affected facilities in parentheses) were August 6, 2018 (1), August 28, 2018 (1), September 4, 2018 (1), and September 5, 2018 (1).


• Allowance of event notification days in basis window. Cadmus excluded event notification days

from consideration for the basis window when calculating customer baselines. This exclusion

was justified because Cadmus’ analysis of load impacts on notification days in the PY9

evaluation suggested that many customers increased or decreased their loads in response to

event notifications. The ICSP did not exclude event notification days when calculating customer

baselines.

• Different treatment of estimated readings. PPL Electric Utilities estimated about 1% of all

hourly interval readings for participating facilities on event or weekdays that were not holidays

or notification days between April 1, 2018, and September 15, 2018. Cadmus replaced these

estimated readings with missing values and did not include them in the analysis sample.

• Different methods for calculating customer baselines. To the extent possible, the ICSP

attempted to align its baseline calculation method with Cadmus’ method. However, for all small

C&I facilities, 90% of GNE facilities, and 20% of large C&I facilities, Cadmus employed regression

analysis to calculate the baseline whereas the ICSP employed day-matching. The ICSP employed

day-matching because it is transparent and easier for participants to understand than

regression. Cadmus used regression after determining it yielded more accurate savings

estimates than day-matching.


Table 17-5 shows the verified PYTD and P3TD demand savings, which were calculated by analyzing

individual participant facility loads and estimating savings for individual facilities during each event hour.

Cadmus averaged the PY9 and PY10 estimates of verified demand reduction for individual events to

calculate the Phase III (P3VTD) program impacts.

Table 17-5. PYTD and P3TD Demand Response Program Savings Summary

Savings Type Demand (MW)

PYRTD 111.3

PYVTD Gross 111.5

PYVTD Net (3) -

P3RTD (1) 112.7

P3VTD Gross (2) 116.6

P3VTD Net (3) - (1) Savings are calculated as the average of demand reductions for the July 2, July 3, August 6, August 28, September 4, and September 5 Act 129 events in 2018. (2) Savings are calculated as the average of the demand reductions for individual Act 129 demand response events in PY9 and PY10. (3) There are no net savings because neither free riders nor spillover apply to this program. C&I and GNE participants are not expected to curtail their loads without notification of PPL Electric Utilities system peaks and without compensation.




The process evaluation assessed program implementation and customer satisfaction. The main research

objectives focused on these areas:

• Event implementation successes and challenges

• Customer response to event notifications and the event enrollment process

• Customer response to events and participation challenges, especially with back-to-back events

• Customer satisfaction with the incentive amount, the ICSP, and the overall program


The PY10 process evaluation activities for the Demand Response Program featured interviews with PPL

Electric Utilities and ICSP program managers and online surveys of participants.

Table 17-6 lists the process evaluation sampling strategy. The process evaluation’s survey activity did not

count participants in the same way as the impact evaluation. The impact evaluation counted the

number of customer facilities that participated in at least one event in PY10 (n=60 facilities). For the

survey as part of the process evaluation, a participant was defined as an enrolled company contracted

by the ICSP (n=25 unique companies which had 64 facilities). This company did not have to participate in

an event in PY10 to qualify for the survey, but it did have to have been enrolled for the PY10 program

and received the event notifications.

Table 17-6. PY10 Process Evaluation Sampling Strategy

Stratum Stratum

Boundaries Mode

Population Size

Assumed Proportion


Target Sample

Size

Achieved Sample

Size

Number of Records

Selected for Sample

Frame (1)




Staff Telephone in-depth Interview

2 N/A 2 2 2 N/A

Participant Surveys

Enrolled Companies Contracted by CPower

Online survey

25 (3) N/A 12 12 25 100%

Program Total 27 N/A 14 14 27 N/A (1) Sample frame is the enrolled customer companies with contact information that were asked to complete the survey. The final sample frame includes unique records in the PPL Electric Utilities tracking database. (2) Percent contacted means the percentage of the sample frame that were emailed to complete surveys. (3) There were 25 unique companies contracted by CPower, the ICSP, that enrolled in the PY10 Demand Response Program. Cadmus included enrolled companies that did not participate in any events in its survey population. Cadmus did not survey the companies under contract with the demand response aggregators NRG and Direct Energy because it did not have customer contact information. The survey’s population count of participants, therefore, differs from the impact evaluation’s participant count. The impact evaluation counts as participants the number of customer facilities that participated in at least one event.



In early November 2018, Cadmus interviewed the program managers from PPL Electric Utilities and the

ICSP. The interviews covered program operations, event implementation, and event performance

outcomes as well as any program changes, areas working well, and areas experiencing challenges.


Between mid-November and early December 2018, Cadmus contacted all 25 enrolled companies, even if

they did not participate in any events, to ask them to complete an online survey.124

The email was directed to the person who authorized the events at each company, typically an energy

manager. The survey sought 12 completes out of the 25 companies, with no subquotas based on

customer segment or level of event participation to ensure that survey responses were representative

of all.

Cadmus coordinated with PPL Electric Utilities program staff and key account managers and the ICSP on

the survey. The ICSP sent an email notifying enrolled customers of the survey one day before Cadmus

sent the invitation email. One week before the survey closed, PPL Electric Utilities’ key account

managers emailed the remaining customers who had not yet responded to encourage completion of the

survey.

Table 17-7 lists total contacts, the outcome (final disposition) of each record, and response rate.

Table 17-7. PY10 Demand Response Participant Survey Sample Attrition Table

Description of Online Survey Outcomes Count

Population (number of CPower, NRG, and Direct Energy enrolled facilities) 64

Removed: NRG and Direct Energy contracted facilities 4

Removed: Duplicate facility contacts 35

Sample Frame (number of unique companies) 25

Survey Sample Frame (used for online surveys) 25

Not started 13

Opted out 0

Partial complete (not included in survey findings analysis) 0

Completed Surveys 12

Response Rate (completed surveys divided by number of records) 48%

Because of the small number of respondents (n=12), the expected confidence and precision levels for

survey data are not reported here. Therefore, data gathered from the participant surveys should be

viewed more qualitatively than quantitatively.

124 Cadmus did not survey the four enrolled customers under contract with the demand response aggregators NRG and Direct Energy because it did not have customer contact information.




In PY10, PPL Electric Utilities and the ICSP successfully implemented six events, including two pairs of

back-to-back events. This was twice as many events as in PY9, which had one pair of back-to-back

events. The Demand Response Program recruited four new companies in PY10 and retained around 90%

of the participants from PY9. PPL Electric Utilities and the ICSP operated the program the same as in PY9.

The Demand Response Program’s successful event implementation and strong performance can be

attributed to three factors:

• Having a familiar and clear set of operational procedures. PPL Electric Utilities, the ICSP, and

participating customers were prepared to handle the greater number of events in PY10 because

operational procedures were kept the same as in PY9. Moreover, in early June 2018 the ICSP

held a seasonal readiness webinar to educate any new participating customers and remind

repeat participating customers of the event procedures and expectations.

• Knowing which participating customers could fill in load performance gaps. PPL Electric

Utilities expressed concern regarding one of its large capacity customers and its ability to meet

load reductions if this customer was not able to participate in an event or deliver on its enrolled

load expectation. The ICSP addressed this concern by reviewing the operations and previous

event performance of customers and identifying the ones that could compensate for the

underperformance of a large capacity customer. The ICSP acted on this information in PY10

when one of the large capacity customers was not able to deliver.

• Oversubscribing the number of participating customers. As a performance gap backup plan,

the ICSP enrolled more customers than the program needed to meet the capacity projections.

Rather than place customers on a program wait list, the ICSP added any interested, qualified

customers. These additional customers could provide the additional MW load reduction needed

should a large capacity customer not be able to deliver.

PPL Electric Utilities and the ICSP noted one challenge in PY10: the two pairs of back-to-back events

occurred on a Monday and following a Monday holiday, which meant event notifications were sent out

on a Sunday and on Labor Day Monday, respectively. PPL Electric Utilities believed this timing would

inconvenience participating customers. However, that was not the case. Participating customers

responded to the event notifications and enrolled in events, albeit event enrollment was slower than on

a normal weekday.

17.4.3.2 Participant Profile

Of the 25 enrolled companies (contracted by CPower, the ICSP), 80% had one facility enrolled in the

PY10 program, 68% were manufacturing facilities, 52% participated in all six events, and 84%

participated in back-to-back events. As shown in Table 17-8, the online surveys captured a fairly

representative sample of enrolled companies. The 12 survey respondents represented approximately

53% of the 111.5 MW average peak load reduction in PY10.


Table 17-8. PY10 Demand Response Enrolled Company and Survey Respondent Profile

Characteristic All Enrolled Customers

(Population n=25) Surveyed Customers

(Sample n=12)

One Facility vs. Multiple Facilities

Customer had one facility enrolled in the program 80% 67%

Customer had multiple facilities enrolled in the program 20% 33%

Facility Type

Manufacturing Facility 68% 75%

School/University 12% 8%

Office 8% 0%

Retail 8% 8%

Medical/Health 4% 8%

Event Participation Count

Six Events 52% 58%

Five Events 16% 17%

Four Events 4% 0%

Three Events 8% 8%

Two Events 4% 8%

One Event 0% 0%

Zero Events 16% 8%

Participation in Back-to-Back Events

Yes 84% 92%

No 16% 8%

Note: All percentages based on analysis of customer and facility data provided by the ICSP.

17.4.3.3 Event Notifications and Enrollment

Most respondents were satisfied with the timing of event notifications and the online event enrollment

process. Nine of the 12 respondents were very satisfied with the amount of time between the

notifications and the start of the events. Eight of 11 respondents (one did not answer the question)

were satisfied with the online event enrollment process; seven said they were very satisfied and one

was somewhat satisfied. Figure 17-2 shows respondents’ satisfaction with the timing of event

notifications and the online event enrollment process.

For these two items, the survey did not ask respondents who said they were less than satisfied to

explain their reasons. Instead, the survey asked everyone for suggestions on ways to improve the event

notifications and the online event enrollment process. Only one respondent offered a suggestion. This

respondent disliked having to go through the event enrollment process more than once a week and

suggested a one-time enrollment instead of having to enroll in each event individually.


Figure 17-2. Satisfaction with Timing of Event Notifications and Online Event Enrollment Process

Source: Survey question, “CPower notified you in advance of upcoming PPL Act 129 Program Events.

You should have received a notification between 10:10 a.m. and noon on days before events. How

satisfied were you with the amount of time between the advance notification and the start of the

event?” and “You enrolled in events and specified the hours of participation through CPower's

online website. How satisfied were you with the online event enrollment process?”

17.4.3.4 Event Experience and Participation Challenges

In general, most respondents found it easy to participate in the PY10 events. Of 11 respondents (the

respondent who did not answer the question did not participate in any events), three respondents said

it was very easy and five said it was somewhat easy to participate. In contrast, most respondents found

it difficult to participate in the back-to-back events. Seven said it was somewhat difficult and one said it

was very difficult. Table 17-9 shows the number of respondents who said it was easy or difficult to

participate in general and back-to-back events.

Table 17-9. Ease/Difficulty of Participating in PY10 Events

Events in General (n=11) Vs. Back-to-Back Events (n=11)

3 Very easy 1

5 Somewhat easy 2

0 Neither 0

2 Somewhat difficult 7

1 Very difficult 1

Source: Survey question, “How easy or difficult was it for your facility/facilities to

participate in the PPL events this summer?” and “How easy or difficult was it for your

facility/facilities to participate in back-to-back PPL events this summer?”

The survey asked those respondents who participated in fewer than six events why their facilities were

unable to participate in all events. Of the six respondents asked this question, three said not having

enough benefits to outweigh the costs, two said there were too many interruptions to business

operations, and one said an event had coincided with the annual facility shutdown.


Similarly, the survey asked respondents what was difficult about participating in the back-to-back

events. Of 10 respondents who answered, six said the back-to-back events impacted their production

and three said occupant comfort was affected from shortening HVAC runtimes.

When asked what would make it easier to participate in events, six of eight respondents said increasing

the amount of the incentive. Other suggestions were providing more communication outside of event

days and having access to historical meter data.

17.4.3.5 Participant Satisfaction

In PY10, eight of 12 respondents were satisfied with the Demand Response Program—five were very

satisfied and three were somewhat satisfied. One respondent who was not too satisfied did not provide

a reason. Responses to other questions revealed this respondent’s dissatisfaction with the timing of

event notifications, the online event enrollment process, the incentive amount, and the ICSP. Despite

reporting dissatisfaction with the program, this respondent’s company nonetheless participated in all six

events.

Figure 17-3 compares overall satisfaction with the program in PY9 and PY10. In PY9, eight of 10

respondents were satisfied; in PY10, eight of 12 respondents were satisfied. When expressed as a

percentage, satisfaction appears to have decreased from 80% in PY9 to 67% in PY10; however, this may

be misleading because of the small sample sizes. These small sample sizes also means that confidence

and precision of the survey data cannot be estimated. It is possible that any decrease in satisfaction may

be due to the greater number of events in PY10, but because of the small sample sizes, this explanation

cannot be supported with confidence. Another difference is that the PY9 survey was conducted by

telephone and the PY10 survey was online. Each survey mode has its set of biases and strengths, such as

self-selection bias, interviewer bias, and respondent anonymity that can influence responses.

Figure 17-3. Overall Satisfaction with Demand Response Program

Source: Survey question, “How would you rate your overall satisfaction with the Demand Response Program?”


Six of 12 respondents were neither satisfied nor dissatisfied with the incentive amount, and two were

very satisfied and two were somewhat satisfied. Only one respondent was not too satisfied. Note that

when Cadmus administered the surveys, participating customers had not received their incentive

payments, but they had been informed of the amount they would be receiving. Figure 17-4 shows the

response breakdown on satisfaction with the incentive amount.

Figure 17-4. Satisfaction with the Incentive Amount

Source: Survey question, “How would you rate your satisfaction with the incentive

amount you will receive?”

During the staff interviews, PPL Electric Utilities acknowledged the issue with the timing of incentive

payments. Incentive payments are made approximately 90 days after the end of the event season. PPL

Electric Utilities needs this time to review and approve the incentives and for the ICSP to process and

send out the incentives. In the PY9 evaluation report, Cadmus recommended that the ICSP advise

customers when they could expect to receive the incentive payment. The ICSP implemented this

recommendation by specifying the timing of the payment in the customer’s contract. During the

interview, the ICSP noted that in PY10 it received one complaint about the timing of the incentive

payment and worked with this participating customer to resolve it.

In PY10, eight of 12 respondents were satisfied with the ICSP—five were very satisfied and three were

somewhat satisfied. One respondent was not at all satisfied because of difficulties with the event

enrollment website and that the ICSP had not responded to emails in a timely manner. Figure 17-5

shows the response breakdown on overall satisfaction with the ICSP.

Figure 17-5. Overall Satisfaction with the ICSP

Source: Survey question, “Thinking about your interactions with CPower, how would

you rate your overall satisfaction with CPower?”


17.4.4 Cost-Effectiveness Reporting

A detailed breakdown of program finances and cost-effectiveness is presented in Table 17-10. Total

resource cost (TRC) benefits were calculated using gross verified impacts. Per the TRC Order, 75% of the

customer incentive payment is used as a proxy for the participant cost when calculating the TRC ratio for

the program. PYTD values represent PY10 costs and benefits, and P3TD values represent phase costs and

benefits up to PY10. Net present value (NPV) PYTD costs and benefits are expressed in PY10 dollars. NPV

costs and benefits for P3TD financials are expressed in PY8 dollars.


Table 17-10. Summary of Demand Response Program Finances – Gross Verified

Row #

Cost Category PYTD ($1,000) P3TD ($1,000) (6)



3 Participant Costs (net of incentives/rebates paid by utilities) ($470) ($633)


EDC CSP EDC CSP



7 Marketing (4) - - - -

8 Program Delivery (5) - $833 - $1,871



11(6) Program Overhead Costs (Sum of rows 5 through 10) (1), (6),

(10) $874 $2,105


- -

13 Total NPV TRC Costs (Net present value of sum of rows 4, 11, and 12) (1), (7)

$2,283 $4,004

14 Total NPV Lifetime Electric Energy Benefits - -


16 Total NPV Lifetime Operation and Maintenance (O&M) Benefits

- -


18 Total NPV TRC Benefits (8) (Sum of rows 14 through 17) (8), (1) $5,059 $9,622

19 TRC Benefit-Cost Ratio (9) 2.22 2.40 (1) May not sum to total due to rounding. (2) All costs for Plan Design and Development are portfolio level costs and are assigned to customer sectors at the end of the phase. These portfolio costs are not assigned to specific programs. (3) Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. (4) Includes the marketing ICSP and marketing costs by program ICSPs. (5) Includes ICSP rebate processing, direct program management, customer support, technical assistance to customers, site visits, legal, QA/QC documentation. These costs cannot be quantified separately and are included as “Program Delivery” costs. (6) P3TD amounts are discounted back to PY8. (7) Total TRC Costs includes Total EDC Costs and Participant Costs. (8) Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include: avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. (9) TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. (10) Total costs include those incurred for PY10 after the Semi-Annual Report filed Jan. 15, 2019.



Overall, in PY10 the Demand Response Program exceeded the Act 129 compliance target of 92 MW for all event hours by 21% and is on track to

meet the Act 129 projected demand reduction for Phase III. Most participating customers were satisfied with the timing of the event

notifications, the online event enrollment process, the ICSP, and the program overall.


Conclusion 1: PPL Electric Utilities and the ICSP

successfully reduced peak demand in PY10 as the

program met its per-event compliance target and

remains on track to exceed the Phase III

compliance target of 92 MW.

Findings Support

• The program achieved an average peak load reduction of 111.5 MW in PY10, exceeding the Act 129

compliance target of 92 MW for all event hours. For Phase III, the program achieved an average peak load

reduction of 116.6 MW, putting the program on track to exceed the Act 129 compliance target.

• The program met its per-event compliance target of at least 78.2 MW, or 85% of the total 92 MW

compliance target, in each of the six events (see Figure 17-1).

Conclusion 2: PPL Electric Utilities and the ICSP

had a resilient program in PY10 that overcame

participation and capacity adversities by

exercising the backup plan in place.

Findings Support

• In PY10, PPL Electric Utilities, the ICSP, and participants experienced six events, including two pairs of back-

to-back events. This was twice as many events as in PY9, which had only one pair of back-to-back events (see

section 17.4.3.1 Program Delivery).

• PY10 had fewer participating facilities, 60 facilities compared to 93 in PY9; nevertheless, the program still

met the Act 129 compliance target (see Table 17-2).125

• PY10 had lower enrolled demand response capacity (124.0 MW) than PY9 (141.8 MW) and still met the Act

129 compliance target. 126

• The ICSP had a load performance backup plan in place on the chance one of the large load capacity

customers was unable to participate in an event. The backup plan involved enrolling more customers in the

program than needed and identifying which of the enrolled customers could make up the difference of a

large load capacity customer (see section 17.4.3.1 Program Delivery).

125 Cadmus. Annual Report to the Pennsylvania Public Utility Commission. November 15, 2018. Prepared for PPL Electric Utilities.

126 Ibid.


Conclusion 3: Despite participants’ reporting

difficulty with back-to-back event participation,

the program achieved the per-event compliance

target for the two pairs of back-to-back events

and observed no event fatigue or low program

satisfaction.

Findings Support

• The two pairs of back-to-back events occurred on a Monday and following a Monday holiday, which meant

event notifications were sent out on a Sunday and Labor Day Monday. PPL Electric Utilities expressed

concern that back-to-back events would inconvenience the participating customers (see section 17.4.3.1

Program Delivery).

• Eight of 11 participating customers reported finding it difficult to participate in the back-to-back events. Of

the 10 respondents who explained the difficulties, six said the events impacted production and three said the

events impacted occupant comfort (see section 17.4.3.4 Event Experience and Participation Challenges).

• Customers who participated in fewer than six events gave these reasons for opting out of the event: three

said not enough benefits to outweigh the costs, two said too many interruptions to business operations, and

one said that an event coincided with the annual facility shutdown (see section 17.4.3.4 Event Experience

and Participation Challenges).

• Participants exceeded the 78.2 MW per-event compliance target for the two pairs of back-to-back events. On

average, participants reduced 106.8 MW and 109.7 MW on July 2 and July 3, respectively, and 116.4 MW

and 104.1 MW on September 4 and September 5, respectively (see Figure 17-1).

• Eight of 12 respondents were satisfied with the program—five were very satisfied and three were somewhat

satisfied. One respondent was not too satisfied, yet the company still participated in all six events (see

section 17.4.3.5 Participant Satisfaction).

Table 17-11. Status of Recommendations for the Demand Response Program

Demand Response Program





Conclusion 1: PPL Electric Utilities and the ICSP successfully

reduced peak demand in PY10 as the program met its per-event

compliance target and remains on track to exceed the Phase III

compliance target of 92 MW.

Maintain the robust backup plan for a variety of

participation and capacity scenarios to manage

program risks and challenges.

Implemented.

Conclusion 2: PPL Electric Utilities and the ICSP had a resilient

program in PY10 that overcame participation and capacity

adversities by exercising the backup plan in place.

Conclusion 3: Despite participants’ reporting difficulty with back-

to-back event participation, the program achieved the per-event

compliance target for the two pairs of back-to-back events and

observed no event fatigue or low program satisfaction.

Consider providing customers with year-to-year

performance results and a historical summary of

past events on the ICSP’s online event enrollment

website.

Being considered.

Appendix A. Upstream Lighting Cross-Sector Sales PPL Electric Utilities | A-1

Upstream Lighting Cross-Sector Sales

A.1.1 Introduction

The Efficient Lighting Program is intended to serve residential customers. However, because PPL Electric

Utilities pays incentives directly to manufacturers, the actual participants are not known. Owners of

small businesses are assumed to comprise a portion of customers who buy discounted LEDs from

participating retailers. LEDs installed in commercial settings are subject to different assumptions set

forth by the PA TRM, affecting annual savings. To determine the proportion of program LEDs purchased

by commercial customers, Cadmus conducted a study called a cross-sector sales analysis.

For PY10 sector-specific reported savings and incentive expenditures, PPL Electric Utilities used results

from the PY8 cross-sector sales analysis, when Cadmus last conducted a cross-sector sales analysis.

Cadmus used PY10 results to make ex post adjustments to quantities and savings by sector to determine

the realization rate. PPL Electric Utilities will adjust its reported savings proportions and expenditure

accounting in PY11.

A.1.2 Surveys

Cadmus used data from the residential and small business general population surveys it conducted in

June 2019 and PPL Electric Utilities’ customer records to estimate the proportions of sales attributable

to residential and commercial customers. Details regarding survey sampling and methodology can be

found in Section 10.5.2.1 General Population Surveys and Appendix R Survey Methodology.

Cadmus used self-reported information from the general population surveys to establish assumptions

used to estimate the proportion of bulbs purchased in each sector. In its surveys, Cadmus asked

respondents to report various details about their LED purchase experience:

• If they purchased LEDs in the six months prior to the survey

• How many LEDs they purchased

• From which retailer they purchased the LEDs

• For small business owners:

▪ If their organization is responsible for lighting purchases

▪ A description of their business type

▪ Whether they installed any LEDs in residential applications (or facilities they build/manage)

Cadmus used this information to estimate the percentage of residential and small business customers

who purchased LEDs from participating retailers and the average number of LEDs they purchased. These

two estimates, when multiplied by each sector’s customer base, establishes a theoretical number of

LEDs purchased in the six months prior to the survey. Although self-reported LED sales are not expected

to be highly accurate (mainly because of recall bias about when respondents thought they made the

purchase), such bias is expected to be similar between both residential and small business populations.

Therefore, a relative proportion of LEDs purchased can be reasonably derived from these estimates.


Cadmus made an additional adjustment to the small commercial base to account for the proportion of

surveyed respondents who indicated their business was not responsible for purchasing lighting for their

facility. This adjustment amounts to a default assumption that the landlords or other parties who

purchase lighting for those commercial customers’ facilities do not purchase equipment from retailers

that participate in the Efficient Lighting Program. Therefore, Cadmus considers this adjustment, which

reduced the assumed small business customer base by 9%, to be conservative.

To ensure the appropriateness of applying metrics gleaned from the small commercial customer survey

to PPL Electric Utilities’ small commercial customer base, Cadmus compared the distribution of standard

industrial classification (SIC) codes and annual kWh usage in the survey respondent group to those in the

assumed population.127 In doing this, Cadmus found the distributions to be reasonably similar. Cadmus

also compared these metrics among commercial survey respondents not responsible for lighting

decisions in their facilities to those who were responsible and did not observe material differences.

Lastly, Cadmus compared the resulting metrics to the PY8 results and found that only the percentage of

residential purchasers was statistically significantly different (31% in PY10 vs. 24% in PY8).128

The computed metrics and resulting proportions are shown in Table A-1.

Table A-1. PY10 Cross-Sector Proportion Calculations by Sector

Population

Percentage of Respondents Purchasing LEDs from Participating

Retailers

Number of LEDs per Customer


Customer Base: [c]

LEDs Purchased from

Participating Retailers:

[a]*[b]*[c](1)

Cross-Sector

Proportion

Estimate: [a] n Estimate: [b] n

Small Commercial 17% 387 15.89 61 91,473 250,388 6%

Residential 31% 300 10.17 92 1,200,000 3,744,000 94%

Total 687 153 1,291,473 3,994,388 100%

(1) Totals may not match manually computed totals due to rounding of inputs. This is a theoretical number, based on survey responses, of all screw-in LEDs purchased from participating retailers and is used only to derive the proportions by sector. Since awareness of the buy-downs is relatively low, Cadmus does not limit its analysis to only those survey respondents who knew they had purchased program bulbs. This should not skew the results because these customers are buying LEDs from participating retailers, and Cadmus expects that the likelihood of purchasing a program vs. a non-program LED is based on factors such as stocking, promotions, etc., not sector.

To compute a statistical confidence interval around the proportions shown in Table A-1, Cadmus ran

simulations of the above computations, treating the distribution of the number of LEDs per respondent

as a normally distributed random variable and the percentage of LED purchasers as a uniform random

variable.129 The true cross-sector proportion lies between 4.7% and 8.2%, or about 1.86% precision at

85% confidence.

127 These metrics are included in PPL Electric Utilities’ customer information system, from which an extract is provided quarterly to Cadmus.


129 The normal distribution was truncated at zero to ensure the number of bulbs was positive.


A.1.3 Savings Inputs and Impact

In the PA TRM, assumptions regarding HOU and CF vary by sector. For the residential sector, these

assumptions are deemed. However, for the commercial sector, HOU and CF assumptions vary by

building type, as described in Table 3-5 of the PA TRM. To account for this, Cadmus collected self-

reported building types from small business general population survey respondents who recently

purchased LEDs from participating retailers and mapped these to the list of building types in the TRM.

Cadmus then calculated commercial sector average HOU and CF values weighted by the number of LEDs

sold that are attributable to each building type.

Table A-2 shows cross-sector sales assumptions updated to reflect PY10 evaluation findings.

Table A-2. PY10 Cross-Sector Assumptions by Sector

Population Delta Watts

(∆W)

Hours of Use

per day (HOU)

Coincidence

Factor (CF)

In-Service

Rate (ISR)

Interactive

Effect: kWh

Interactive

Effect: kW

Residential 34 3.00 0.106 92% 0.940 1.120

Small Commercial 34 6.07 0.409 98% 1.000 1.192

A.1.4 Calculating Uncertainty for Program Savings

When calculating the cross-sector sales proportion, Cadmus accounted for population sizes, the

proportion of survey respondents who purchased LEDs, and the average number of LEDs respondents

purchased at participating retailers. Cadmus used observed survey means and variances to simulate

100,000 realizations of both the number of LEDs purchased and the proportion of LED purchasers in

each sector. For each realization, Cadmus calculated a unique cross-sector sales proportion, which it

used to adjust energy savings, then estimated the uncertainty of the adjusted energy savings based on

variation in the realizations.

Cadmus estimated precision for energy and demand savings by calculating the 7.5 and 92.5 percentiles

of the distribution Cadmus used to determine the confidence interval around the total energy savings

and to report precision. Based on the variance in the realizations, Cadmus estimated precision for

energy and demand savings at 1.86% with 85% confidence.

Appendix B. Site Inspection Summary PPL Electric Utilities | B-1

Site Inspection Summary

Table B-1 summarizes programs receiving verification site visits by Cadmus or the ICSP (listed in column “Inspection Firm”). The table includes

the numbers of inspections, and resolution of discrepancies.

Table B-1. Site Inspection Summary

Program Inspection Firm Inspections Conducted

Sites with Discrepancies from

Reported Values Summary of Common Discrepancies

Non-Residential Sector

Custom

Warren Energy Engineering, LLC (for Cadmus)

41 10 • Discrepancies only found on small sample sites (10 of 10 small sample site visits) though all equipment and quantities matched reported values

CLEAResult (the ICSP)

76 76

• Submitter estimate of original savings was not accurate • Actual metered data used in place of estimates • Project not modeled accurately originally compared to installed condition • Project scope deviated from original submission

Efficient Equipment Prescriptive Lighting

Warren Energy Engineering (for Cadmus)

18 11 • Implementer calculated incorrect annual lighting hours of use and coincidence factor calculated from metered logger data, reported incorrect pre and post fixture type and quantity, space cooling type, heating fuel type and pre and post fixture wattages


240 123

• Wrong HOU given on Appendix C form vs. what was found from customer interviews on site • Wrong number of lights submitted on application • Wrong amount of bulbs in the fixtures submitted • Incorrect wattage selected for baseline fixtures

Efficient Equipment Direct Discount Lighting


7 5 • Implementer reported incorrect building type (and associated TRM HOU), pre and post fixture quantity, pre and post fixture wattages, space cooling type, heating fuel type, and pre and post fixture wattages


234 55

• Wrong number of lights submitted on application • Wrong amount of bulbs in the fixtures submitted • Projects started before receiving preapproval • Integrated fixtures not used in application • Projects over 120,000 switched from prescriptive to customer provided HOU

Efficient Equipment HVAC

Cadmus 5 3 • Facility type, equipment efficiencies, baseline controls, equipment capacities

Efficient Equipment Motors

Cadmus 5 2 • Equipment quantities, equipment efficiencies, equipment horsepower





Efficient Equipment Refrigeration

Cadmus 6 0 None

Efficient Equipment Other

Cadmus 1 0 None

Efficient Equipment CLEAResult (the ICSP)

22 12

• Project savings may have increased or decreased as a result of site visits which made the projects switch from prescriptive to customer submitted HOU • Ineligible equipment removed from applications • Project scope deviated from original submission

Midstream Lighting


288 N/A • Product Installation not complete • Building type changes


15 (74 projects)

15

• TRM Building type • Verified quantities • Building operating hours • Building coincidence factor • Baseline equipment wattage • Facility space conditioning

Residential Sector

Energy Efficient Home – New Homes Component

Performance Systems Development (PSD) (for the ICSP)

57 51

• Lighting (27) – All lighting discrepancies involved an incorrectly reported percentage of energy-efficient bulbs. Raters often miscount or fail to identify all the existing fixtures in the home, causing inconsistencies in reporting • Appliances (17) – Appliance discrepancies were most often caused by misreported equipment efficiency ratings • Domestic Hot Water (16) – Domestic Hot Water discrepancies were most often caused by misreported efficiency ratings • Cooling Equipment (14) – Cooling Equipment discrepancies were most often caused by misreported efficiency ratings

Low-Income Sector

WRAP Manufactured Homes

CMC (the ICSP) 15 10 • Missed opportunities were installed during inspections and technician improvement plan created for the technician involved. Monthly inspection reports provided to all agencies/technicians for areas of improvement

WRAP Master-Metered Multifamily (all job types)

CMC (the ICSP) 125 0 None

WRAP Baseload CMC (the ICSP) 554 179 • Missed opportunities were installed during inspections and technician improvement plan created for the technician involved. Monthly inspection reports provided to all agencies/technicians for areas of improvement





WRAP Low Cost CMC (the ICSP) 194 63 • Missed opportunities were installed during inspections and technician improvement plan created for the technician involved. Monthly inspection reports provided to all agencies/technicians for areas of improvement

WRAP Full Cost CMC (the ICSP) 0 0 • Full cost audits are not performed under this program

Appendix C. Home Energy Report Impact Evaluation Detail PPL Electric Utilities | C-1

Home Energy Report Impact Evaluation Detail

C.1 Methodology

C.1.1 Data Preparation

Cadmus worked with PPL Electric Utilities and the ICSP to acquire the data necessary for the Home

Energy Education Program evaluation in PY10. Major data preparation steps included cleaning and

compiling the program tracking data, billing consumption and weather data, and testing for significant

differences in annual pretreatment consumption between treatment and control customers, by wave.

Cadmus received program tracking data from the ICSP and billing consumption from PPL Electric

Utilities. This section describes the steps Cadmus took to process the data and verify customers in the

tracking and billing data.

Program Tracking Data

Cadmus received Home Energy Education Program tracking data from the ICSP at the close of PY10.

These data included treatment group customers who received home energy reports in the current or a

previous year and control group customers tracked since the program’s inception. Because the Home

Energy Education Program was implemented as a randomized control trial, Cadmus included all of the

possible customers in its evaluation, adopting a “once in, always in” policy for customers originally

randomized into either the treatment or control group prior to the launch of the home energy reports.

Cadmus verified customer program data from the ICSP with the program tracking data it collected in

previous program years to account for any customers not included in the ICSP’s tracking data.

Table C-1 shows customer attrition through PY10, by treatment and control groups, by wave, and as

originally randomized and active at the beginning of treatment in PY10. The attrition process captures

customers whose accounts closed (became inactive) since the launch of the program and accounts who

stopped receiving home energy reports.

Table C-1. PY10 Customer Attrition

Wave Originally Randomized

Active at the Beginning of Treatment in PY10

Treatment Control Treatment Control

Legacy Wave 1 50,000 50,000 33,239 33,278

Legacy Wave 2 55,040 25,003 38,697 17,484

Expansion Wave 1 48,711 12,653 38,161 9,883

Low-Income Wave 1 73,500 18,560 47,006 12,055

Low-Income Wave 2 21,401 10,046 12,288 5,807

Phase III Expansion Wave 1 30,584 12,234 26,318 10,569

Program Total 279,236 128,496 195,669 89,076


Billing Data

Cadmus collected customer billing data for each wave from PPL Electric Utilities to supplement the

billing data it had collected and cleaned in previous program years. To clean the billing data, Cadmus

followed these steps:

1. Dropped customers whose accounts went inactive before the delivery of the first energy reports

2. Cleaned and calendarized bills, which included dropping bills that covered more than 65 days,

dropping bills with negative consumption, dropping bills earlier than one year prior to the

delivery of the first energy reports, and truing up bills with estimated reads

3. Dropped customers with less than 11 months of pretreatment bills

Table C-2 provides the attrition in the PY10 analysis sample from data cleaning steps. The final modeling

sample included customers in Cadmus’ final tracking data who were not dropped during the billing data

cleaning process and were included in the billing analysis. These customers were not necessarily active

at the beginning of treatment in PY10.

Weather Data

Cadmus collected weather data from the weather station closest to each home and estimated the

heating degree days (HDDs) and cooling degree days (CDDs) for each customer billing cycle. After

merging the weather and billing data, Cadmus allocated the billing cycle electricity consumption, HDDs,

and CDDs to calendar months.

Table C-2. PY10 Sample Attrition from Data Cleaning

Step in Attrition Legacy Wave 1 Legacy Wave 2 Expansion Wave 1

Treatment Control Treatment Control Treatment Control

Originally Randomized Customers 50,000

(100%)

50,000

(100%)

55,040

(100%)

25,003

(100%)

48,711

(100%)

12,653

(100%)

Active at Program Launch 49,494

(99%)

49,465

(99%)

54,233

(99%)

24,618

(98%)

48,089

(99%)

12,499

(99%)

Less than 11 Months of

Pretreatment Data

47,792

(96%)

47,793

(96%)

50,422

(92%)

22,866

(91%)

47,317

(97%)

12,296

(97%)

Final Modeling Sample 47,792

(96%)

47,793

(96%)

50,422

(92%)

22,866

(91%)

47,317

(97%)

12,296

(97%)

Step in Attrition Low-Income Wave 1 Low-Income Wave 2 Phase III Expansion Wave 1

Treatment Control Treatment Control Treatment Control

Originally Randomized Customers 73,500

(100%)

18,560

(100%)

21,401

(100%)

10,046

(100%)

30,584

(100%)

12,234

(100%)

Active at Program Launch 73,157

(100%)

18,467

(99%)

20,964

(98%)

9,818

(98%)

28,356

(93%)

11,356

(93%)

Less than 11 Months of

Pretreatment Data

71,752

(98%)

18,106

(98%)

20,514

(96%)

9,575

(95%)

26,770

(88%)

10,718

(88%)

Final Modeling Sample 71,752

(98%)

18,106

(98%)

20,514

(96%)

9,575

(95%)

26,770

(88%)

10,718

(88%)


C.1.2 Verification of Balanced Treatment and Control Groups

Cadmus verified that subjects in the randomized treatment and control groups were equivalent in

pretreatment energy use, as it does every year. Cadmus conducted the random assignment of eligible

customers to treatment or control groups for Legacy Wave 2 in Phase I, Expansion Wave 1 and Low-

Income Waves 1 and 2 in Phase II, and the Expansion Wave 1 in Phase III. The ICSP performed the

randomization for Legacy Wave 1. Cadmus verified the equivalence of waves using the cleaned billing

data, comparing preprogram average annual consumption from before the launch of the program.

Table C-3 provides the PY10 results of the tests for significant differences in treatment and control group

pretreatment consumption. Cadmus found that all waves were balanced. No statistically significant

differences existed between the pretreatment consumption of treatment and control groups in any

wave.

Table C-3. PY10 Tests for Significant Differences in Annual Pretreatment Consumption

Wave

Customers Average Annual Electricity Use

per Customer (kWh/yr) p-value (1)

Treatment

Group

Control

Group

Treatment

Group

Control

Group Difference

Legacy Wave 1 47,692 47,725 18,518 18,459 58.71 0.1971

Legacy Wave 2 50,226 22.768 27,591 27,652 104.10 0.1455

Expansion Wave 1 47,210 12,265 23,221 23,214 6.98 0.8983

Low-Income Wave 1 71,599 18,066 11,868 11,817 50.83 0.3720

Low-Income Wave 2 20,312 9,485 8,036 8,111 75.37 0.4571

Phase III Expansion Wave 1 26,020 10,426 15,159 15,163 3.85 0.9384

(1) A p-value >0.05 indicates an insignificant difference at the 5% significance level.

C.1.3 Ex Post Verified Savings Methodology

Energy Savings Model Specification

Cadmus used regression analyses of monthly billing data from customers in the treatment and control

groups to estimate the Home Energy Education Program’s energy savings. The billing analysis conformed

to IPMVP Option C, whole facility,130 and the approach described in the Uniform Methods Project.131,132

130 Efficiency Valuation Organization. International Performance Measurement and Verification Protocol, Concepts and Options for Determining Energy and Water Savings, Volume 1. January 2012. Page 25. (EVO 10000 – 1:2012) Available online: http://www.evo-world.org/

131 Agnew, K., and M. Goldberg. Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, Chapter 8: Whole-Building Retrofit with Consumption Data Analysis Evaluation Protocol. U.S. Department of Energy, National Renewable Energy Laboratory. April 2013. (NREL/SR-7A30-53827) Available online: http://www1.eere.energy.gov/office_eere/de_ump_protocols.html

132 Stewart, J., and A. Todd. Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, Chapter 17: Residential Behavior Protocol. U.S. Department of Energy, National Renewable

http://www.evo-world.org/

http://www1.eere.energy.gov/office_eere/de_ump_protocols.html


Methods also followed those described in the Phase III Evaluation Framework for behavioral

programs.133

More specifically, Cadmus used a multivariate regression to analyze the energy use of customers who

had been randomly assigned to treatment and control groups. Cadmus tested and compared two

general model specifications to check the robustness of savings results:

• The post-only model regresses customer average daily consumption on a treatment indicator

variable and includes as regressors customers’ pretreatment energy use, month-by-year fixed

effects and weather.134 The model is estimated only with posttreatment customer bills.

• The difference-in-differences (D-in-D) fixed effects model regresses average daily consumption

on a treatment indicator variable, month-by-year fixed effects, customer fixed effects, and

weather. The model is estimated with pretreatment and posttreatment customer bills.

Both models yielded savings estimates that were within each other’s confidence intervals, meaning that

their results were not statistically different (illustrated in Figure C-1 and Figure C-2 later in this section).

In PY10, Cadmus reported the results of the post-only model, consistent with previous Phase III program

years.

The error terms of the post-only model and D-in-D fixed effects model should be uncorrelated with

program participation (𝑃𝐴𝑅𝑇𝑖) and other observable variables because of the random assignment of

homes to treatment and control groups, and therefore ordinary least squares should result in an

unbiased estimate of the average daily savings per customer. Cadmus clustered the standard errors on

customers to account for arbitrary correlation in customer consumption over the analysis period.

The following sections provide additional details about each modeling approach.

Post-Only Model

The post-only model was specified assuming the average daily consumption (𝐴𝐷𝐶𝑖𝑡) of electricity of

home ‘𝑖’ in month ‘𝑡’ as given by Equation C-1.

Equation C-1

𝐴𝐷𝐶𝑖𝑡 = 𝛽1𝑃𝐴𝑅𝑇𝑖 ∗ 𝑃𝑌𝑡 + 𝛽2𝑃𝑟𝑒– 𝑈𝑠𝑎𝑔𝑒𝑖 + 𝛽3𝑃𝑟𝑒– 𝑆𝑢𝑚𝑚𝑒𝑟𝑖 + 𝛽4𝑃𝑟𝑒– 𝑊𝑖𝑛𝑡𝑒𝑟𝑖 +

𝛽5𝑃𝑟𝑒– 𝑈𝑠𝑎𝑔𝑒𝑖 × 𝜏𝑡 + 𝛽6𝑃𝑟𝑒– 𝑆𝑢𝑚𝑚𝑒𝑟𝑖 × 𝜏𝑡 + 𝛽7𝑃𝑟𝑒– 𝑊𝑖𝑛𝑡𝑒𝑟𝑖 × 𝜏𝑡 + 𝑊′𝛾 + 𝜏𝑡 + 𝜀𝑖𝑡

Energy Laboratory. August 2014. (NREL/SR-7A40-62497) Available online: http://www1.eere.energy.gov/office_eere/de_ump_protocols.html

133 Pennsylvania Public Utility Commission. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, and Demand Side Analytics, LLC. Final version May 8, 2018. See Behavior Section 6.1.1.

134 Allcott, H., and T. Rogers. “The Short-Run and Long-Run Effects of Behavioral Interventions: Experimental Evidence from Energy Conservation.” American Economic Review 104 (10), 3003-3037. 2014.

http://www1.eere.energy.gov/office_eere/de_ump_protocols.html


Where:

𝛽1 = Coefficient representing the conditional average treatment effect of the

program on electricity use (kWh per customer per day).

𝑃𝐴𝑅𝑇𝑖 = Indicator variable for program participation (which equals 1 if customer ‘𝑖’ was

in the treatment group and 0 otherwise).

𝑃𝑌𝑡 = Indicator variable for each program year (which equals 1 if the month ‘𝑡’ was in

the program year and 0 otherwise).

𝛽2 = Coefficient representing the conditional average effect of pretreatment

electricity use on posttreatment average daily consumption (kWh per customer

per day).

𝑃𝑟𝑒– 𝑈𝑠𝑎𝑔𝑒𝑖 = Mean household energy consumption of customer ‘𝑖’ across all pretreatment

months.

𝛽3 = Coefficient representing the conditional average effect of pretreatment summer


per day).

𝑃𝑟𝑒– 𝑆𝑢𝑚𝑚𝑒𝑟𝑖 = Mean household energy consumption of customer ‘𝑖’ during June, July, August,

and September of the pretreatment period.

𝛽4 = Coefficient representing the conditional average effect of pretreatment winter


per day).

𝑃𝑟𝑒– 𝑊𝑖𝑛𝑡𝑒𝑟𝑖 = Mean household energy consumption of home ‘𝑖’ during December, January,

February, and March of the pretreatment period.

𝑊 = Vector using both HDD and CDD variables to control for the impacts of weather

on energy use.

𝛾 = Vector of coefficients representing the average impact of weather variables on

energy use.

𝜏𝑡 = Average energy use in month ‘𝑡 reflecting unobservable factors specific to the

month. The analysis controls for these effects with month-by-year fixed effects.

𝛽5 = Coefficient representing the conditional average effect of pretreatment

electricity use, given month ‘𝑡’, on posttreatment average daily consumption

(kWh per customer per day).

𝛽6 = Coefficient representing the conditional average effect of pretreatment summer



𝛽7 = Coefficient representing the conditional average effect of pretreatment winter



𝜀𝑖𝑡 = Error term for customer ‘𝑖’ in month ‘𝑡.’


Difference-in-Differences Fixed Effects Model

The D-in-D fixed effects model was specified assuming the average daily consumption (𝐴𝐷𝐶𝑖𝑡) of

electricity of customer ‘𝑖’ in month ‘𝑡’ as given by Equation C-2:

Equation C-2

𝐴𝐷𝐶𝑖𝑡 = 𝛼𝑖 + 𝜏𝑡 + 𝑊′𝛾 + 𝛽1𝑃𝐴𝑅𝑇𝑖 × 𝑃𝑂𝑆𝑇𝑡 + 𝜖𝑖𝑡

Where:

𝛽1 = Coefficient representing the conditional average treatment effect of the

program on electricity use (kWh per customer per day).

𝑃𝐴𝑅𝑇𝑖 = Indicator variable for program participation (which equals 1 if customer ‘𝑖’ was

in the treatment group and 0 otherwise).

𝑃𝑂𝑆𝑇𝑡 = Indicator variable for whether month ‘𝑡’ is pre- or posttreatment (which equals

1 if month ‘𝑡’ was in the treatment period and 0 otherwise).

𝑊 = Vector using both HDD and CDD variables to control for the impacts of weather

on energy use.

𝛾 = Vector of coefficients representing the average impact of weather variables on

energy use.

𝛼𝑖 = Average energy use in customer ‘𝑖’ reflecting unobservable, non-weather-

sensitive, and time-invariant factors specific to the customer. The analysis

controlled for these effects with customer fixed effects.

𝜏𝑡 = Average energy use in month ‘𝑡’ reflecting unobservable factors specific to the

month. The analysis controlled for these effects with month-by-year fixed

effects.

𝜖𝑖𝑡 = Error term for customer ‘𝑖’ in month ‘𝑡’

Regression Analysis Estimates

Cadmus estimated separate treatment effects for each wave and program year. Table C-4 shows both

the D-in-D fixed effects model and post-only model estimates of average daily savings per customer, by

wave and program year. All of the models were estimated by ordinary least squares, and Huber-White

robust standard errors were adjusted for correlation over time in a customer’s consumption.


Table C-4. Treatment Effects for the Home Energy Education Program by Model Specifications

Treatment Year

Legacy Wave 1 Legacy Wave 2 Expansion Wave

D-in-D Fixed Effects

Post-Only D-in-D Fixed

Effects Post-Only


Post-Only

PY1 -0.224** (0.085)

-0.176** (0.067)

N/A N/A N/A N/A

PY2 -0.694***

(0.046) -0.696***

(0.028) N/A N/A N/A N/A

PY3 -0.926***

(0.047) -0.905***

(0.029) -0.941***

(0.074) -1.016***

(0.046) N/A N/A

PY4 -1.011***

(0.048) -1.016***

(0.029) -1.199***

(0.075) -1.268***

(0.047) N/A N/A

PY5 -0.882***

(0.049) -0.900***

(0.03) -1.179***

(0.076) -1.237***

(0.048) N/A N/A

PY6 -0.849***

(0.049) -0.858***

(0.031) -1.205***

(0.078) -1.291***

(0.05) -0.562 (0.089)

-0.583 (0.06)

PY7 -0.856***

(0.050) -0.835***

(0.031) -1.063***

(0.079) -1.126***

(0.051) -0.743***

(0.081) -0.688***

(0.05)

PY8 -0.879***

(0.051) -0.874***

(0.032) -0.987***

(0.080) -1.091***

(0.052) -0.822***

(0.083) -0.745***

(0.052)

PY9 -0.889***

(0.052) -0.877***

(0.033) -0.91*** (0.081)

-1.047*** (0.053)

-0.657*** (0.085)

-0.614*** (0.053)

PY10 -0.914***

(0.052) -0.904***

(0.034) -0.894***

(0.083) -1.035***

(0.055) -0.704***

(0.087) -0.600***

(0.055)

Treatment Year

Low-Income Wave 1 Low-Income Wave 2 Phase III Expansion Wave 1


Post-Only D-in-D Fixed

Effects Post-Only


Post-Only

PY6 -0.068 (0.054)

-0.104** (0.035)

N/A N/A N/A N/A

PY7 -0.394***

(0.051) -0.399***

(0.03) -0.101 (0.064)

-0.106** (0.033)

N/A N/A

PY8 -0.36*** (0.053)

-0.370*** (0.032)

-0.132* (0.069)

-0.140*** (0.036)

-0.126* (0.072)

-0.119** (0.041)

PY9 -0.236***

(0.055) -0.246***

(0.034) -0.058 (0.073)

-0.046 (0.039)

-0.313*** (0.073)

-0.325*** (0.042)

PY10 -0.277***

(0.057) -0.290***

(0.036) -0.056 (0.077)

0.008 (0.042)

-0.545*** (0.074)

-0.538*** (0.043)

Standard errors clustered on customers are presented below the estimated treatment effect in parentheses (*** Significant at 1%; ** Significant at 5%; * Significant at 10%). The treatment effects represent the average daily savings per treatment group customer.

The PY10 savings estimates from the D-in-D fixed effects and post-only models were statistically

indistinguishable, suggesting that the estimated treatment effects do not depend on the modeling

approach. Cadmus reported savings based on the post-only models for all waves because of the

increased precision achieved with these models; this is seen in the smaller standard errors of post-only

estimates compared to D-in-D fixed effects estimates.

PY10 post-only treatment effects were significant across all waves and program years with one

exception. Cadmus evaluated average daily savings per customer of 0.008 (p-value of 0.8496) kWh for

Low-Income Wave 2. A p-value less than 0.10 suggests that the estimate is not statistically different

from 0.0 kWh/day, which may mean that either customers in this wave truly did not reduce their

consumption compared to the control group or savings in these periods were too small to identify with


the available sample size. Table C-2, provided earlier, shows that the counts of treatment and control

customers in this wave are considerably smaller than in the other waves. Results are similar to PY9,

when Cadmus could not distinguish savings from 0.0 kWh/day. As a result Cadmus recommended

discontinuation of the wave in PY9. The program currently plans to discontinue treatment for Low-

Income Wave 2 in PY11 and PY12.

Table C-5 shows the estimated average daily savings as a percentage of control group consumption, by

program year and wave. Consistent with the previous year, Legacy Wave 1, Legacy Wave 2, and

Expansion Wave 1 maintained consistent savings through PY10 and continued to achieve the largest

percentage savings ranges of all waves, with savings ranging between 1.0% (Expansion Wave 1) to 1.9%

(Legacy Wave 1). Low-Income Wave 1 savings increased slightly from PY9 after not receiving treatment

for much of PY8.

Encouragingly, the Phase III Expansion Wave 1 continued to ramp up in its third year of treatment,

achieving savings of 1.2%. Though slower to ramp up than other general residential waves, customers in

the Phase III Expansion Wave reached expected levels of savings, compared to baseline energy

consumption, in PY10.

Table C-5. Percentage Treatment Effects by Model Specifications

Treatment

Year

Legacy Wave 1 Legacy Wave 2 Expansion Wave 1

D-in-D Fixed

Effects Post-Only

D-in-D Fixed

Effects Post-Only

D-in-D Fixed

Effects Post-Only

PY1 0.54%**

(0.205%)

0.425%**

(0.161%) N/A N/A N/A N/A

PY2 1.332%***

(0.089%)

1.335%***

(0.054%) N/A N/A N/A N/A

PY3 1.921%***

(0.098%)

1.877%***

(0.059%)

1.403%***

(0.11%)

1.514%***

(0.069%) N/A N/A

PY4 2.013%***

(0.095%)

2.024%***

(0.058%)

1.676%***

(0.105%)

1.772%***

(0.066%) N/A N/A

PY5 1.715%***

(0.094%)

1.751%***

(0.058%)

1.572%***

(0.102%)

1.649%***

(0.064%) N/A N/A

PY6 1.711%***

(0.099%)

1.729%***

(0.062%)

1.661%***

(0.107%)

1.779%***

(0.068%)

0.801%

(0.127%)

0.831%

(0.085%)

PY7 1.864%***

(0.109%)

1.819%***

(0.068%)

1.656%***

(0.122%)

1.753%***

(0.079%)

1.368%***

(0.15%)

1.266%***

(0.092%)

PY8 1.88%***

(0.109%)

1.869%***

(0.069%)

1.511%***

(0.123%)

1.671%***

(0.08%)

1.485%***

(0.151%)

1.346%***

(0.094%)

PY9 1.892%***

(0.11%)

1.866%***

(0.07%)

1.346%***

(0.12%)

1.55%***

(0.079%)

1.149%***

(0.149%)

1.075%***

(0.094%)

PY10 1.924%***

(0.11%)

1.903%***

(0.071%)

1.318%***

(0.122%)

1.525%***

(0.081%)

1.218%***

(0.15%)

1.038%***

(0.096%)


Treatment

Year

Low-Income Wave 1 Low-Income Wave 2 Phase III Expansion Wave 1

D-in-D Fixed

Effects Post-Only

D-in-D Fixed

Effects Post-Only

D-in-D Fixed

Effects Post-Only

PY6 0.193% (0.154%) 0.297%**

(0.099%) N/A N/A N/A N/A

PY7 1.372%***

(0.176%)

1.392%***

(0.104%)

0.513%

(0.325%)

0.542%**

(0.166%) N/A N/A

PY8 1.236%***

(0.183%)

1.27%***

(0.11%)

0.696%*

(0.365%)

0.737%***

(0.19%)

0.296%*

(0.169%)

0.281%**

(0.097%)

PY9 0.8%***

(0.187%)

0.835%***

(0.114%)

0.313%

(0.391%)

0.246%

(0.207%)

0.717%***

(0.167%)

0.746%***

(0.096%)

PY10 0.915%***

(0.189%)

0.961%***

(0.118%)

0.298%

(0.413%)

-0.043%

(0.224%)

1.225%***

(0.167%)

1.209%***

(0.098%) (1) Standard errors are presented below the estimated treatment effect in parenthesis (*** Significant at 1%; ** Significant at

5%; * Significant at 10%).

Annual Program Energy Savings

Cadmus estimated program savings in PY10 for each wave’s population of treated customers as the

product of average daily savings per participant and the number of days these customers were treated

in PY10, shown in Equation C-3. Cadmus assumed that the ICSP intended to treat all eligible customers

at least once in PY10 and included treatment days for customers who should have received treatment in

PY10 (i.e., those who were still active and randomized as a treatment customer), even when customers

were not explicitly flagged as receiving PY10 treatment.

Equation C-3

𝑆𝑎𝑣𝑖𝑛𝑔𝑠ℎ = −�̂�1,ℎ ∗ ∑ 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐷𝑎𝑦𝑠𝑖,ℎ

𝑁

𝑖=1

Where:

�̂�1,ℎ = Average daily savings (kWh) per treatment group customer in wave ‘ℎ’,

estimated from Equation C-1.

𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐷𝑎𝑦𝑠𝑖,ℎ = The number of days customer ‘𝑖’ in wave ‘ℎ’was treated in PY10.

Cadmus estimated realization rates for each wave as the ratio of verified program savings to reported

program savings (estimated by the ICSP).

Table C-6 shows the estimate of PY10 total savings and average annual savings per customer with 85%

confidence intervals for each wave. Except for the second Low-Income Wave, the 85% confidence

intervals do not include zero, suggesting customers in these waves, on average, achieved savings

significantly greater than 0 MWh/year. The 85% confidence intervals contain the reported program total

savings (42,079 MWh/year), suggesting the two estimates are not significantly different.


Table C-6. PY9 Home Energy Education Program Savings Estimate

Wave Point Estimate (MWh/yr) 85% Confidence Interval

(Lower Bound) 85% Confidence Interval

(Upper Bound)

Legacy Wave 1 10,740 10,160 11,319

Legacy Wave 2 14,304 13,212 15,396

Expansion Wave 8,074 7,001 9,147

Low-Income Wave 1 4,725 3,888 5,562

Low-Income Wave 2 -33 -287 220

Phase III Expansion Wave 5,019 4,435 5,603

Total Program (1) 42,829 40,883 44,774


Ex Post Verified Savings across Time

Figure C-1 through Figure C-7 provide the percentage daily savings across time for each pre- and post-

treatment month through PY9. Cadmus calculated the percentage daily savings for each wave as the

ratio of average daily savings to monthly average control group consumption. The figures report the

post-only results for each wave, with the monthly percentage savings and confidence intervals (gray)

resulting from the D-in-D fixed effects model (blue) plotted to show pre-treatment consumption trends.

The green line in the figures shows the monthly savings resulting from the reported post-only model

specifications. The post-only monthly savings trend closely to the D-in-D fixed effects monthly savings,

and they remain within the D-in-D fixed effects confidence interval across months and waves; this

suggests that the savings estimated by each model specification are not significantly different. It also

suggests that savings are robust and not dependent on the model specification (pre-post versus D-in-D

fixed effects). For every wave, the confidence interval in the pre-treatment period contains zero. This

suggests that treatment and control groups had equivalent consumption prior to treatment.

Cadmus specified both the D-in-D fixed effects and post-only models with month and year fixed effects.

To avoid linear dependency in the regressors, Cadmus dropped one month and year from each model

specification. In the D-in-D fixed effects model specifications, Cadmus dropped the last month prior to

treatment, which explains the gap in monthly savings in each figure for this month. Similarly, Cadmus

dropped the first month of treatment in the post-only model specifications (since they did not include

pre-treatment bills).

Figure C-1 and Figure C-2 show steady savings across months in PY10 for Legacy Wave 1 and Legacy

Wave 2. A slight upward trend is observed for Legacy Wave 1 that began after the Phase II ICSP resumed

treatment in October 2014 and that persisted throughout Phase II. Legacy Wave 1 savings appear

consistent through Phase III. Legacy Wave 2 month-to-month saving are more variable than Legacy

Wave 1, and its savings appear to be slightly lower in Phase III compared to Phase II. Monthly savings

reflect actual weather, so small changes in savings from year to year may not be program-related.


Figure C-1. Percentage Daily Savings across Time: Legacy Wave 1

Figure C-2. Percentage Daily Savings across Time: Legacy Wave 2

Figure C-3 shows savings for Expansion Wave 1 increased until the beginning of Phase III and have since

slightly diminished. This is consistent with the trends of Legacy Wave 2, where savings reached a steady

state but have since slightly declined.


Figure C-3. Percentage Daily Savings across Time: Expansion Wave 1

Figure C-4 shows that similar to Legacy Wave 1 and Expansion Wave 1, Low-Income Wave 1 savings

ramped up through Phase II and have slightly declined since.

Figure C-4. Percentage Daily Savings across Time: Low-Income Wave 1

Figure C-5 shows the percentage daily savings by month for Low-Income Wave 2. The confidence

intervals around monthly savings are wider for this wave than any other wave and include zero savings

for most months, which is consistent with Cadmus’ finding that savings in PY10 were statistically

insignificant and imprecisely estimated.


Figure C-5. Percentage Daily Savings across Time: Low-Income Wave 2

Figure C-6 shows the percentage daily savings by month for the Phase III Expansion Wave. The monthly

percentage daily savings have increased since treatment began, finishing PY10 with savings of 1.2%. This

is expected for a newer wave that is still ramping up savings.

Figure C-6. Percentage Daily Savings across Time: Phase III Expansion Wave 1

To compare savings trends across waves, Cadmus provides Figure C-7, which shows percentage daily

savings by the number of months since first treatment for each wave. Within the first year of treatment,

savings peaked between 1% and 2% for each wave. Legacy Wave 1 and Legacy Wave 2 savings ramped


quicker than other waves in their first year of treatment, and they continue to generate the highest

savings. Expansion Wave 1 and Low-Income Wave 1 ramped up at similar rates and both show slight

decreases in savings the last two years. The Low-Income Wave 2 and Phase III Expansion Wave ramped

up more inconsistently than the other waves, but Phase III Expansion Wave is now saving at a similar

rate to other waves, whereas Low-Income Wave 2 is not generating consistent savings. Again, savings

reflect changes in weather, which can explain some differences in savings by months of treatment.

Figure C-7. Percentage Average Daily Savings from First Month of Treatment

Demand Reduction Evaluation Methodology

As in PY9, Cadmus did not evaluate demand reductions using hourly data in PY10. Instead, it converted

each wave’s PY9 average energy savings into demand reductions using the evaluated PY4 ratio of peak

demand reduction values to average per-customer energy savings per hour. Across Legacy Wave 1 and

Legacy Wave 2, Cadmus estimated average per-customer demand reductions of 0.041 kWh/hr and

0.056 kWh/hr for each wave, or 193% and 108% of each wave’s average per-customer energy savings

per hour, respectively. Cadmus used the weighted average of these ratios (148%) to convert PY10

program energy savings into demand reductions, assuming ratios stayed constant through time, and

allowing demand reductions to be scaled by energy savings observed in PY10.

Note that the definition of peak demand changed between PY4 and PY10. In PY4, peak demand was

calculated for the top 100 hours of PPL Electric Utilities’ system demand. In PY10, peak hours are

defined as hours with day-ahead forecasts for the PJM market that are 95% or more of the PJM peak

summer forecast.

C.1.4 Uplift Analysis Methodology

Savings from the Home Energy Education Program reflected both behavioral changes, such as turning

off lights in unoccupied rooms and adjusting thermostat settings, and investments in energy-efficient

products, such as high-efficiency furnaces and LEDs. In PY9, some customers who installed efficiency


products because of home energy reports may have received rebates from PPL Electric Utilities through

other Act 129 programs. Customers could also have received rebates in previous program years

following receipt of their first home energy report, and these efficiency products could have continued

to save energy into PY9. In these cases, the Home Energy Education Program billing analysis would

capture the savings from these products, causing them to be counted in both the Home Energy

Education Program and PPL Electric Utilities’ other efficiency programs.

To avoid double-counting of cross-program savings generated by the home energy report program,

Cadmus subtracted cross-participation savings from the residential portfolio savings. To do this, Cadmus

conducted an uplift analysis to estimate the impacts of the Home Energy Education Program on

participation in PPL Electric Utilities’ residential and low-income efficiency programs and the energy

savings from that participation. Cadmus refers to any difference in the rate of participation and savings

as participation uplift and savings uplift.

The following sections provide details on uplift results.

Cross-Participation in Downstream Residential Rebate Programs

Cadmus used the experimental design of the Home Energy Education Program to estimate home energy

report savings from PPL Electric Utilities’ efficiency program participation.

To illustrate, suppose that there is an equal number of customers in the treatment and control groups

and that the utility markets the benefits of installing Product A to all residential customers. Customers in

the treatment and control groups will receive the same marketing and be eligible for incentives from the

utility for Product A. The impact of energy reports on adoption of Product A can then be estimated as

the difference in adoption of Product A—and savings—between the randomized treatment and control

groups. Any differences can be attributed to the home energy report program.

For products and services promoted by utility programs and tracked at the customer level (downstream

programs), Cadmus estimated the participation and savings uplift by matching Home Energy Education

Program treatment and control customers in each wave to the energy efficiency program participation

tracking data in PPL Electric Utilities’ tracking database, starting in the month when treatment began

through to the end of PY10.135

Home Energy Education Program treatment and control customers participated in 9 downstream PPL

Electric Utilities rebate programs from PY2 through PY10. These were the Appliance Recycling Program,

Energy Efficiency Kits and Education Program,136 Energy Efficient Home Program, Low-Income WRAP,

Non-Residential Energy Efficiency Program, Renewable Energy Program, Residential Energy Assessment

135 Each product’s record in PPL Electric Utilities’ tracking database includes the program to which it belongs, along with the date the product was installed. Cadmus’ database records the evaluated ex post annual savings.

136 Formerly named the E-Power Wise Program.


and Weatherization Program, Residential Home Comfort Program, and Residential Retail Program

(equipment component).


After matching tracking data to Home Energy Education Program customers, Cadmus calculated

participation uplift. Cadmus defined participation uplift as the difference in the percentage of treatment

group customers participating in at least one rebate program and the percentage of control group

customers participating in at least one rebate program.

The control group’s participation rate captured the business-as-usual effect of marketing and word-of-

mouth impacts on customers’ participation in other PPL Electric Utilities’ Act 129 programs. This

baseline participation rate is defined as the number of control group customers who participated in at

least one other Act 129 program in PY10, divided by the total number of control group customers. The

home energy reports had an additive effect on participation in the other programs if the cross-program

participation rate was greater for treatment customers than it was for control customers.

Table C-7 shows the PY10 participation rate uplift results for each wave of the Home Energy Education,

broken out by program. Cadmus first provides the differences in rates of cross-participation between

treatment and control groups (uplift participation) then the percentage uplift participation relative to

control group participation. There is no participation uplift for the Non-Residential Energy Efficiency

Program for Expansion Wave 1, Low-Income Wave 1, and Low-Income Wave 2, since no control group

customers participated in PY10. In PY0, the treatment customers only installed lighting measures from

the Mid-stream Commercial Lighting and Non-residential Prescriptive Lighting programs. Roughly 18

unique customers installed 34 measures. The Non-Residential Energy Efficiency Program contributed

897 MWh to total uplift.

Table C-7. Participation Uplift by Program (Per 1,000 Customers)

Program

Participation Uplift per 1,000 Customers (Percentage Participation Uplift)

Legacy Wave 1

Legacy Wave 2

Expansion Wave 1

Low-Income Wave 1

Low-Income Wave 2

Phase III Expansion

Wave 1

Appliance Recycling -0.08 0.99 0.86 0.12 0.22 1.84

-0.74% 8.04% 8.51% 1.90% 4.64% 19.28%


0.94 0.79 -1.83 0.13 0.40 0.28

35.79% 13.65% -27.41% 0.52% 2.24% 7.09%

Energy Efficient Home 1.22 0.40 1.32 -0.10 1.05 3.46

8.55% 2.05% 8.12% -1.53% 33.90% 23.59%

Low-Income WRAP -0.60 0.08 0.16 -2.93 0.65 -0.58

-19.91% 5.45% 10.99% -9.12% 1.91% -19.68%


-0.03 0.04 0.05 0.06 0.08 N/A

-0.02

-100.00% 35.58% N/A N/A -19.68%


Savings Uplift

The savings uplift analysis followed a simple-differences approach. Similar to the approach suggested in

the Behavior Section of the Phase III Evaluation Framework,137 Cadmus followed these steps to estimate

uplift savings from downstream programs:

1. Matched the program tracking data for each program year to the treatment and control

customers by a unique identifier

2. Assigned each transaction to a month based on the participation date field in the tracking data

3. Excluded any installations that occurred prior to the customer being assigned to the treatment

or control group

4. Calculated the average monthly electricity savings of each efficient product installed by a Home

Energy Education customer, proportioned across months by the accrued heating and cooling

degree days in each month for products sensitive to weather (Cadmus proportioned annual

savings across months equally for products not sensitive to weather). Cadmus used the ex post

gross verified savings for each product in PPL Electric Utilities’ tracking database.

5. Summed the monthly average savings, by customer, for all products installed prior to a given

month through the end of PY10. Cadmus incorporated customer inactive dates and measure

lives of products when aggregating monthly savings

6. Calculated the average annual savings accrued per customer for the treatment and control

groups during PY10

7. Calculated the incremental average annual savings per customer from other programs by taking

the difference in annual per-customer savings for the treatment group and control group

Multiplying the incremental average annual savings per customer by the number of program customers

treated in PY10 yielded the estimate of the total Home Energy Education Program savings from

participation in other PPL Electric Utilities energy efficiency programs and counted by the other

efficiency programs.

Table C-8 provides the results of the savings uplift analysis by program. The largest proportion of cross-

program savings came from the Appliance Recycling Program, which saved across all waves except Low-

Income Wave 2.

137 Pennsylvania Public Utility Commission. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, and Demand Side Analytics, LLC. Final version May 8, 2018. See Behavior Section 6.1.1.8.


Table C-8. Savings Uplift by Program

Program

Total Uplift Savings (MWh/yr)

(Percentage of Program Total Savings)

Legacy Wave 1

Legacy Wave 2

Expansion Wave 1

Low-Income Wave 1

Low-Income Wave 2

Phase III Expansion

Wave 1

Appliance Recycling 391.86 483.86 228.41 413.91 -27.91 90.17

3.65% 3.38% 2.83% 8.76% 83.69% 1.80%


-8.19 75.74 5.88 32.56 15.79 -9.48

-0.08% 0.53% 0.07% 0.69% -47.35% -0.19%

Energy Efficient Home -7.29 404.14 -277.47 -20.32 44.12 214.63

-0.07% 2.83% -3.44% -0.43% -132.29% 4.28%

Low-Income WRAP -75.22 31.44 37.59 -192.43 0.68 -29.04

-0.70% 0.22% 0.47% -4.07% -2.03% -0.58%

Non-Residential Energy Efficiency (lighting measures)

318.52 210.86 210.95 62.02 -28.43 147.00

2.97% 1.47% 2.61% 1.31% 85.24% 2.93%

Renewable Energy Program* -51.38 116.16 -68.80 1.27 -27.26 167.46

-0.48% 0.81% -0.85% 0.03% 81.75% 3.34%

Residential Energy Assessment and Weatherization*

146.23 266.88 146.81 -7.78 33.12 160.11

1.36% 1.87% 1.82% -0.16% -99.29% 3.19%

Residential Home Comfort* -3.41 -174.67 136.96 31.05 -18.31 212.16

-0.03% -1.22% 1.70% 0.66% 54.89% 4.23%

Residential Retail* 50.20 12.66 -36.60 -9.44 -0.56 -54.50

0.47% 0.09% -0.45% -0.20% 1.69% -1.09%

* Program is no longer running in Phase III.

Appendix D. Evaluation Detail – Efficient Equipment Program PPL Electric Utilities | D-1

Evaluation Detail – Efficient Equipment Program

D.1 Gross Impact Evaluation – Lighting

D.1.1 Methodology

Evaluation Sampling Approach

In PY9, Cadmus evaluated the prescriptive and the direct discount lighting projects as separate strata. In

PY10, Cadmus grouped the direct discount and the prescriptive lighting projects into one stratum.

Cadmus calculated an annual sample size for these lighting projects to meet the evaluation

requirements described in the Phase III Evaluation Framework.138 The PY10 evaluation sampling plan

was designed to meet 90% confidence and ±10% precision (90/10) for the lighting stratum because

lighting is a high-impact measure, contributing the majority of savings to the program and to the Non-

Residential sector portfolio.

Cadmus’ PY8 sampling plan used 90/10 with the error ratio set to equal 0.50 because lighting

contributed over 80% of the kWh/yr savings for the total portfolio. This estimate was adjusted in PY9,

based on the average of error ratios observed in PY7 and PY8.139 In PY10, based on stratum level CVs

observed in PY9, a minimum sample size of 10 was required per stratum to achieve the targeted

precision at 90/10; this would ensure sufficient representation of the population when error ratios and

calculated sample sizes were small. The Evaluation Framework requires evaluating all projects with ex

ante annual savings greater than 750,000 kWh/yr. Cadmus evaluated all lighting projects (prescriptive

and direct discount) below the threshold with a basic level of rigor and all lighting projects at or above

the threshold with an enhanced level of rigor, as stipulated in the PA TRM.140

Table D-1 shows the PY10 sampling plan by quarter for a final sample size of 40 projects.


139 Cadmus considered the error ratios observed in PY5 and PY6 but found they were inconsistent with the observed error ratios in more recent PY7 and PY8 and thus decided to use the more recent years’ results only.

140 Table 1-2 in the PA TRM defines the thresholds for end-use categories that must be reviewed with enhanced levels of rigor.


Table D-1. Efficient Equipment Program (Lighting) Sampling Strategy

Quarter Population

Size (1)

Target Levels of

Confidence &

Precision

Target

Sample

Size (2)

Achieved

Sample

Size

Evaluation Activity

Q1 215

90/10

~10 15

Site visits and Desk audits Q2 212 ~10 13

Q3 319 ~10 7

Q4 214 ~10 5

Total 960 90/10 ~40 40

(1) Population size refers to the number of unique project job numbers. (2) Sample size was set at the program level, then allocated to strata according to Neyman routine. Each stratum does not

have a target sample size.

Cadmus used a stratified ratio estimation approach to sampling because it is more efficient than using

simple random sampling and results in smaller sample sizes. Cadmus divided all lighting projects into

four substrata: small, medium, large and threshold. These boundaries were established by the

substratum’s contribution to total gross reported kWh/yr savings, following the methods in Chapter 13:

Sampling in The California Evaluation Framework.141 Cadmus determined the number of sample points,

where a point was a job, for each stratum using a Neyman allocation routine that accounts for the

variance in each stratum.142

Table D-2 shows the substrata lighting boundaries for high- and low-energy savings by quarter. In all

quarters, Cadmus verified the census of projects whose ex ante energy savings were greater than the

750,000 kWh/yr threshold, which require enhanced levels of rigor according to the PA TRM. In PY10,

there were 16 threshold lighting participants.

Table D-2. PY10 Quarterly Efficient Equipment Program (Lighting) Substrata Boundaries

Substratum

Q1 Q2 Q3 Q4

kWh /yr

High

kWh/yr

Low

kWh/yr

High

kWh/yr

Low

kWh/yr

High

kWh/yr

Low

kWh/yr

High

kWh/yr

Low

Small 46,152 1,602 65,391 0 104,041 29 91,748 587

Medium 233,772 46,500 176,065 67,576 284,559 106,847 245,543 92,249

Large 746,945 255,581 566,050 176,065 724,901 299,365 531,706 256,182

Threshold 4,300,550 750,000 3,705,369 750,000 2,316,880 750,000 3,096,944 750,000

The PY10 lighting projects were post-stratified at the end of the program year into the final substrata

shown in Table D-3 below. As can be seen, in the breakdown of total participants and reported savings

by final substratum, post-stratification conducted for the final analysis included all projects. Therefore, a

141 TecMarket Works. The California Evaluation Framework. 2004. Pages 368-371.

142 Neyman allocation is a sample allocation method that may be used with stratified samples. The purpose of the method is to maximize survey precision, given a fixed sample size.


project classified as small, for example, when received in Q1 could be reclassified in the post-

stratification.

Table D-3. PY10 Efficient Equipment Program (Lighting) Post-Stratification

Substratum kWh/yr High kWh/yr

Low

Reported

Participants (1)

Reported Savings (MWh/yr)

Percentage Reported Savings

Small 49,796 - 673 11,525 12%

Medium 232,383 49,797 184 22,308 23%

Large 749,999 232,384 87 33,404 35%

Threshold - 750,000 16 28,901 30%

Total 960 96,318(2) 100%(2)

(1) Defined by unique job number. (2) Total does not match sum of rows due to rounding.

Ex Post Verified Savings Methodology for Lighting

The ex post savings incorporated installation rates, adjustments for nonqualifying equipment, and

adjustments for equipment details determined through the sample of projects selected desk audits and

site visits. Cadmus verified installation and qualification rates for all sampled records.

D.1.2 Database Review Findings – Lighting

Cadmus conducted records reviews for 40 lighting projects (across the prescriptive and direct discount

channels). The purpose of the review was to check the database and project data for accuracy and

compliance with the PA TRM requirements. Cadmus verified information recorded in PPL Electric

Utilities’ tracking database by comparing it to corresponding rebate applications, customer-submitted

supporting documentation, and information recorded by the ICSP.

D.1.3 Site Visit and Desk Audit Findings – Lighting

Cadmus conducted site visits for 25 projects in the impact evaluation sample to verify the as-built

conditions for each project and identify any discrepancies reported by the ICSP in the project file.

Cadmus conducted desk audits for the census of 16 threshold lighting projects. Cadmus reviewed logger

data files from lighting hours-of-use measurement devices and the ICSP’s logger data analysis if the ICSP

determined hours of use using metering. The results of this desk audit were combined with the findings

from site visits to determine the verified savings for each of the sampled projects.

If the ICSP’s project documentation and logged or metered data for lighting operating hours were

complete and accurate, Cadmus did not conduct a site visit. If the information in the project

documentation and calculated energy savings could not be fully verified, Cadmus conducted a site visit.

Of the 16 threshold lighting projects, Cadmus conducted three site visits and 13 desk audits. Across the

remaining strata, Cadmus conducted 22 site visits and two desk audits.


If a project had numerous records (approximately 20 or more) in the PA TRM Appendix C Lighting Audit

and Design Tool for Commercial and Industrial Projects,143 Cadmus selected and inspected a sample

using 90/20 criteria for confidence and precision, according to the Phase III Evaluation Framework.144

Cadmus also interviewed facility representatives to determine operating schedules and estimate lighting

hours of use.

Verified savings incorporated site-specific and measure-specific data. Reasons for adjustments to the

ICSP’s reported data included corrections to the following:

• Annual lighting hours of use and

associated coincidence factor calculated

from metered logger data

• Fixture type and quantity

• Lighting control type

• Space cooling type

• Heating fuel type

• Fixture wattage

Overall, the adjustments made to the coincidence factor had the largest impact on verified savings

across the 40 projects. The magnitude of impact from the remaining adjustments for lighting verified

savings was minimal.

D.2 Gross Impact Evaluation – Equipment

D.2.1 Methodology


In PY10, 68 unique customers (billing accounts) completed 72 jobs but two of these jobs involved

equipment installations in multiple strata leaving 70 unique jobs in the population. PPL Electric Utilities

issued rebates for 22 types of equipment. Cadmus evaluated all sampled equipment projects with a

basic level of rigor, according to the Phase III Evaluation Framework.145

The PY10 evaluation sampling plan was designed to meet levels of 85% confidence and 15% precision

(85/15) for the equipment stratum. Cadmus first selected the projects with the largest savings from each

stratum to ensure that a large percentage of the total savings were represented. Cadmus then drew a

simple random sample from each substratum to fill the remaining sample target. The sites where these

143 The PA TRM Appendix C Lighting Audit & Design Tool was designed to document the pre- and post-installation cases of the lighting retrofit and facilitate calculation of energy and demand reductions for large lighting installations.

144 Sampling to meet 90/20 within a facility is based on section 3.3.3.2.3 in the Evaluation Framework for Phase III of the Act 129 Energy Efficiency and Conservation Program. Pennsylvania Public Utility Commission. October 21, 2016.

145 Levels of rigor are described in the Section 3.3.2.2. Pennsylvania Public Utility Commission. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. Prepared by NMR Group, Inc., EcoMetric Consulting, LLC, and Demand Side Analytics, LLC. Final version May 8, 2018.


sampled projects were implemented were reviewed to determine whether additional rebated

equipment had been installed. Cadmus added overlapping projects at the same location to the final

sample.

Cadmus reviewed the sample of 19 project records, which involved verifying information from PPL

Electric Utilities’ tracking database using rebate applications, customer-submitted supporting

documentation, and information recorded by the ICSP to calculate energy savings.

In PY10, Cadmus conducted site visits to verify 17 of the 19 records sampled. Two projects in the random sample were installed at facilities that were permanently closed and Cadmus did not conduct a site visit. Instead, Cadmus conducted a desk audit and applied an in-service rate of zero. Table D-4 presents annual population and sample sizes by substrata.

Table D-4. Efficient Equipment Component (Equipment) Sampling Strategy

Substratum Population

Size (1)

Target Levels of Confidence

& Precision

Target Sample

Size

Achieved Sample

Size Evaluation Activity

HVAC 20

N/A (2) N/A (2)

2 Site visits, desk audit

HVAC – Occupancy Sensors 8 3 Site visits, desk audit

Motors 16 5 Site visits, desk audit

Other 4 2 Site visits, desk audit (3)

Refrigeration 24 7 Site visits, desk audit (4)

Equipment Total 72 85/15 19 19 (1) Population size refers to the number of unique project job numbers per equipment type. (2) Sample size was set at the program level then allocated to strata according to Neyman routine. Each stratum does not have a target sample size.

(3) Cadmus verified one of the Other substratum projects through a desk audit and did not conduct a site visit. (4) Cadmus verified one of the Refrigeration substratum projects through a desk audit and did not conduct a site visit.

Ex Post Verified Savings Methodology for Equipment

The ex post savings incorporated installation rates, adjustments for nonqualifying equipment, and

adjustments for equipment details determined through the sample of projects selected for desk audits

and site visits. Cadmus verified eligibility and installation rates for all sampled records.

D.2.2 Database Review Findings – Equipment

Cadmus conducted a records review of a sample of 19 equipment projects. Cadmus verified records in

PPL Electric Utilities’ tracking database and compared these with corresponding rebate applications,

customer-submitted supporting documentation, and information recorded by the ICSP. The purpose of

the review was to check the database and project data for accuracy and compliance with the PA TRM

requirements.


D.2.3 Site Visit and Desk Audit Findings – Equipment

Cadmus completed site visits and desk audits for 19 unique customers who received rebates for 19

equipment projects. Cadmus verified the as-built conditions for each project and identified

discrepancies in the data reported by the ICSP in the project file. Verified savings incorporated site-

specific data. Reasons for adjustments to the ICSP’s reported data included corrections to the following:

• Facility type

• Equipment quantity

• Equipment efficiency

• Baseline control type

• Equipment capacity

Overall, the factors that had the greatest impact on verified savings across the 19 projects were the

quantity of equipment and in-service rates, followed by reported equipment capacities and baseline

controls.

D.3 Net Impact Evaluation – Lighting and Equipment

D.3.1 Net-to-Gross Ratio Sampling

Table D-5 lists the sampling strategy for the lighting and equipment strata.

Table D-5. PY10 Efficient Equipment Program Lighting and Equipment Stratum Sampling Strategy for Net Savings Research

Stratum Stratum


Size (1)

Assumed Cv or

Proportion in Sample

Design

Assumed Levels of

Confidence & Precision

Target Sample

size


for Sample Frame

Achieved Sample

Size



Equipment Equipment projects

72 0.5 85/15

68

33 7 100%

Lighting (3) Lighting projects

960(4) 0.5 90/10 392 61 100%

(1) Population refers to number of paid projects in PY10. (2) Sample frame is a list of contacts that have a chance to be selected into the sample. Percent contacted means the percentage of the sample frame called to complete surveys. (3) Prescriptive lighting and direct discount lighting combined. (4) Combined population of prescriptive lighting and direct discount lighting participants.

D.3.2 Net-to-Gross Ratio Findings

Free Ridership

Cadmus summed the intention and influence components to estimate the total intention and influence

method free ridership average by stratum, weighted by ex post gross kWh/yr savings. Table D-6

summarizes the intention, influence, and free ridership scores for each stratum. The savings weighted

influence score found 20% of the equipment savings and 0% of the lighting stratum savings could be

classified as free ridership. The savings-weighted average intention scores showed 16% of the

equipment stratum savings and 23% of the lighting stratum savings could be classified as free ridership.


Table D-6. Energy Equipment Program Intention, Influence, and Free Ridership Score by Stratum

Stratum Number of

Respondents Intention Score Influence Score

Free Ridership Score

Equipment 7 16% 20% 36%

Lighting 61 23% 0% 23%

Spillover

The data collected through the surveys did not provide enough information to reliably quantify spillover

in commercial settings; therefore, spillover is reported qualitatively.

Of the lighting stratum respondents, four purchased additional energy-efficient lighting, three

purchased HVAC equipment, three purchased variable frequency drives, one purchased air compressor

equipment, one purchased clothes washing and drying equipment, one purchased insulation, and one

purchased exterior doors after participating in the Efficient Equipment Program. All respondents

credited factors related to PPL Electric Utilities as having some level of influence on their purchasing

decisions.

Of the equipment stratum respondents, one purchased additional energy-efficient lighting after

participating in the Efficient Equipment Program and credited factors related to PPL Electric Utilities as

having some level of influence on their purchasing decision.

Table D-7 shows the NTG ratio results for the equipment and lighting strata of the Efficient Equipment

Program.

Table D-7. PY10 Efficient Equipment Program NTG Ratio Summary

Stratum n Free

Ridership (%)

Spillover (%)


Equipment 7 36% 0% 0.64 46%*

Lighting 61 23% 0% 0.77 12%**

(1) At 85% confidence interval.

(2) At 90% confidence interval.

D.4 Process Evaluation – Lighting and Equipment

D.4.1 Additional Findings

This section presents additional process evaluation findings for the Efficient Equipment Program.

Participant Profile

Cadmus reviewed PPL Electric Utilities’ tracking database and developed a profile of the 1,032 unique

Efficient Equipment Program participants. In PY10, 72 participants received rebates for equipment, 707

received rebates for prescriptive lighting equipment, and 253 received rebates through the direct

discount delivery channel.


Table D-8 shows the survey population and total participant population by sector.

Table D-8. Total and Survey Population by Sector

Sector Total

Population (n=1,032)

Lighting Population

(n=960)

Equipment Population

(n=72)

Survey Respondents

(n=68)

Large C&I 10% 10% 10% 13%

Small C&I 80% 79% 83% 82%

GNE 10% 10% 7% 4%

Residential 0% 1% 0% 0%

Source: PPL Electric Utilities’ tracking database; may not total 100% due to rounding.

Almost half of the survey respondents (48%; n=68) had participated in the Efficient Equipment Program

prior to PY10, and of these 33 respondents, 56% said they worked with a contractor, vendor, or

distributor for their project. Of the respondents who were participating in the Efficient Equipment

Program for the first time in PY10 (n=30), 90% said they worked with a contractor, vendor, or distributor

for their project. The difference in responses between equipment and lighting participants was not

significant. Additionally, 57% of respondents said they did not know about the PPL Electric Utilities’

rebate before interacting with their ESCO, contractor, vendor or other consultant (n=68).

D.4.2 Survey Approach

Survey Frequency

Cadmus conducted online and telephone surveys with PY10 efficient equipment, prescriptive lighting,

and direct discount lighting participants. Where possible, PPL Electric Utilities Key Account Managers

provided outreach for efficient equipment participants to increase response rates. Additional

information can be found in Appendix R Survey Methodology.

Survey Attrition

Participants were first invited to complete the survey via email. Respondents who did not complete the

survey online were then contacted by telephone to complete the survey. Table D-9 lists the total records

used for surveys and the outcome (final disposition) of each record.


Table D-9. Efficient Equipment Program Participant Survey Sample Attrition Table

Description of Call Outcomes Number of Records

Online

Population (number of unique jobs) 1,032

Removed: inactive customer, completed survey in past 3 months, on "opt out" list, selected for a different survey, duplicate contact, on “do not contact” list

607

Removed: incomplete or invalid email address 0

Survey Sample Frame (email invitations sent) 425

Email was returned (bounce back), did not respond, opted-out, or did not complete survey

385

PPL Electric Utilities or market research employee N/A

Cannot confirm project location 3

Did not complete survey 8


Online Response Rate 7%

Telephone

Population (number of unique jobs) 425

Removed: Respondents who completed the survey online, partially completed the survey online, or could not confirm project location.

40

Survey Sample Frame (used for telephone survey calls) 385

Not attempted (1) 42

Records Attempted 343

Not reached: No answer, answering machine, phone busy, refused 302

Screened out: Cannot confirm equipment/not aware of participation, employment, ESCO 1



Telephone Response Rate 10%

Total Completed Surveys (total for all modes) 68

Overall Response Rate 16%

(1) All equipment records were exhausted but the target was reached before all lighting records were needed.

Appendix E. Evaluation Detail – Midstream Lighting Program PPL Electric Utilities | E-1

Evaluation Detail – Midstream Lighting Program

E.1 Gross Impact Evaluation

The Midstream Lighting component reported 23,542 MWh/yr in energy savings. As seen in Figure E-1,

the Small C&I segment contributed to a majority of the savings, followed by the GNE segment.

Figure E-1. Midstream Lighting Program PY10 Reported Energy Savings by Customer Segment

Almost 50% of the total PY10 Midstream Lighting savings across all customer segments may be

attributed to purchases of four-foot linear LED lamps, followed by highbay and lowbay LED fixtures,

which contributed 20% of the savings. Figure E-2 shows the distribution of product reported savings

within each sector.


Figure E-2. Midstream Lighting Program PY10 Reported Product Savings by Customer Segment

E.1.1 Methodology

Evaluation Stratification Approach

In its PY10 evaluation plan for Midstream Lighting, Cadmus planned to sample 46 jobs to meet

confidence and precision of 85/15 at Cv of 0.7. Cadmus aimed to complete 46 verifications consisting of

30 desk audits and 16 site visits. Additionally, Cadmus planned to evaluate sibling jobs that occurred

when sampled jobs were located at sites associated with additional discounted product purchases (jobs)

at different points of time. Finally, Cadmus planned to post-stratify the population and the sample at the

end of PY10 to evaluate program total savings and realization rates. Based on the PY8 and PY9

evaluations, Cadmus recommended post-stratification using the reported annual energy savings of each

job and the distribution of all jobs completed during PY10. Cadmus also recommended an exploratory

analysis after the completion of verification activities to determine if variables besides reported savings

should be incorporated into the stratification.146

Cadmus conducted 14 site visits and 13 desk audits to evaluate 27 randomly sampled jobs and 60

siblings in PY10, for a total of 87 job verifications, and achieved the targeted confidence level of 85%

precision of 15% to report verification findings. Cadmus post-stratified the population for the

Midstream Lighting component using the reported annual energy savings of each job and the

distribution of all jobs completed during PY10. In developing program results, jobs in the non-random




convenience sample stratum did not contribute to the precision calculations for the strata they

belonged to by size definition.

Cadmus determined that savings from high bay/low bay lighting should be assigned to a separate

stratum because reported savings values for high bay/low bay products did not correspond to the

Midstream Lighting Interim Measure Protocol (IMP).147 Additional products installed for jobs with high

bay/low bay lighting were stratified separately by their size category. For example, if a job reported high

bay/low bay lighting and linear fixtures, then the savings for the high bay/low bay lighting were included

in the high bay/low bay stratum, while the savings for the linear fixtures were included in the applicable

size stratum. Sibling jobs were included in the convenience stratum for all products.

Based on exploratory analysis, Cadmus identified reported annual energy savings as the best variable for

stratification and subsequent estimation among the remaining jobs. The population and sample were

post-stratified based on product type and reported annual energy savings (i.e., high bay/low bay lighting

or not, plus four savings categories among non-high-bay lighting jobs) to estimate realization rates,

verified savings, and relative precision.

Table E-1 provides the definitions, population, and sample sizes for all strata.

Table E-1. PY10 Midstream Lighting Impact Evaluation Stratum Definitions and Sample Sizes

Stratum Reported Annual Energy

Savings (kWh) Population Size (2) Sample Size

Midstream Lighting – Convenience N/A(3) 60 60

Midstream Lighting – High Bay/Low Bay N/A(3) 1,273 5

Midstream Lighting – Large 98,075 and more 6 2

Midstream Lighting – Medium 4,671 to 14,011 742 3

Midstream Lighting – Medium-Large 14,012 to 98,074 233 6

Midstream Lighting – Small 4,670 and less 5,416 11

Midstream Lighting Program Total (1) 7,730 87

(1) May not match due to rounding. (2) Population size refers to the number of unique jobs in each stratum. There were 7,633 unique jobs in the program. Jobs

that included high bay/low bay and other lighting products were stratified separately for high bay/low bay products and

for other products. Therefore, the total population of jobs for the impact evaluation is 7,730. (3) This stratum was not limited by size of reported savings.

Table E-2. shows the sampling strategy, target and achieved sample sizes, and the percentage of the

stratum total reported savings that were included in sampled jobs.

147 2016 TRM – Interim Measure Protocol: Lighting Improvements for Midstream Delivery Programs. Version approved January 2019, effective of June 1, 2018 – May 31, 2020.


Table E-2. PY10 Midstream Lighting Impact Evaluation Sampling Strategy

Stratum Target Levels of Confidence

& Precision

Target Sample

Size

Achieved Sample

Size

Percent of Stratum Population

Reported kWh/yr Included in

Sampled Jobs

Evaluation Activity

Midstream Lighting – Convenience N/A N/A 60 100.00%

Record review and either site

visit or desk audit

Midstream Lighting – High Bay/Low Bay

0.7 46

5 2.35%

Midstream Lighting – Large 2 42.06%

Midstream Lighting – Medium 3 0.34%

Midstream Lighting – Medium-Large 6 3.76%

Midstream Lighting – Small 11 0.33%

Midstream Lighting Program Total 87 5.07%

Ex Post Verified Savings Methodology

Records Review

For the impact evaluation-sampled jobs, Cadmus reviewed distributors’ invoices submitted to PPL

Electric Utilities and the ICSP, as well as distributors’ records of sales to the contractor or end-user

purchaser, when available. Cadmus reviewed the technical specification of the reported installed

equipment, confirmed the correct application of the IMP’s baseline and efficient lighting pairing, and the

building type.

Desk Audit and Phone Interview

To calculate verified savings, Cadmus audited 13 jobs and prepared a modified PA TRM Appendix C using

information compiled during the records review and the phone verification interview. Cadmus used the

desk audit phone survey instrument, approved by PPL Electric Utilities and the SWE, to verify the

products installed. The site contact was either the customer or the contractor who purchased and

installed the products for the customer. During the interview, Cadmus confirmed the contact was

familiar with the incentivized purchase and the installed location and verified the quantity of the

reported purchase, building type, hours of use, and space conditioning system with the data in PPL

Electric Utilities’ tracking database to the extent the respondent could provide this information. Cadmus

also gathered information regarding the in situ baseline fixtures and lamps.

A modified Appendix C lighting calculator tool was used for the desk audits that includes columns to

record the verified in-situ baseline fixtures and their wattages, baseline and post-installation quantities,

coincidence factors, hours of use, savings factors, interactive factors, and post-installation in service

rates. Ex post savings were calculated using verified quantities and the independent variables listed in

the IMP, using the methods detailed in the evaluation plan.

Site Visits

To calculate verified savings, Cadmus visited 14 sites to verify 14 jobs and 60 siblings. At the site of each

sampled job, Cadmus reviewed additional sibling jobs associated with the site address for that program

year. These additional jobs could be the same product as the sampled job, which could not be


distinguished from the sampled record. The additional jobs could also be the same or different products

at the site which could be distinguished from the sampled job.

During the site visits, Cadmus verified the building type, and when physical conditions and customer

acceptance allowed, confirmed the independent variables used in the savings algorithms included in the

Midstream IMP. Cadmus used a modified PA TRM Appendix C tool for the inspections that includes

columns to record the observed in situ baseline fixtures and their wattages, observed baseline and post-

installation quantities, evaluated coincidence factors, hours of use, savings factors, interactive factors

and post-installation in service rates. The list of jobs reviewed and verified during the site visit, including

the sampled and sibling jobs, were included in the modified Appendix C. Ex post savings were calculated

using the observed and evaluated values for the independent variables listed in the Lighting

Improvements for Midstream Delivery Programs IMP.

Baseline. Cadmus determined in situ baseline fixtures and lamps wattages through interviews with site

contacts and examination of unchanged, removed, or spare lighting equipment still at the site. In cases

where the baseline could not be verified because the equipment had been removed or the facility

managers did not know the baseline equipment type, or both, Cadmus used the IMP baseline product

that was paired with the new product. If site specific data could be obtained, the baseline equipment

that Cadmus recorded in the PA TRM Appendix C baseline equipment was the in situ equipment, not the

IMP baseline.

In-service rate. During the site visits, Cadmus identified the program-qualified product that replaced the

baseline equipment and verified the fixture or lamp counts, both installed and in storage. Using these

counts, Cadmus calculated the ISR for each job. Often, at the time of site visit, end-use customers or

their contractors were in the process of installing equipment. If, in Cadmus’ judgment, the customer

showed intention to complete the installation and as a result the probability of savings was high,

Cadmus assumed project completion and calculated the ISR.

Building type. Cadmus compared the actual building type to the ICSP’s reported building type to

determine accuracy since hours of use are determined by the building type.

Hours of use. Hours of use were determined using the PA TRM building types for each sampled site.

Cadmus and the ICSP used site-specific hours of use for records with savings greater than 120 MWh/yr.

Additional factors. The independent variables of coincidence factors, interactive energy and demand

factors, and savings factors for lighting controls were based on PA TRM building types and verified by

Cadmus. The presence or absence of any space cooling was determined during the site visit.

Realization rate. Cadmus calculated each site’s realization rate as the ratio of the PA TRM Appendix C

savings calculated with site-specific data to the ex ante savings reported in the PPL Electric Utilities

database for both kWh/yr and kW/yr.


E.1.2 Verification Findings

Overall, Cadmus conducted 27 site visit and desk audit verifications in PY10, for a total of 87 jobs in PPL

Electric Utilities’ tracking database. Table E-3 shows the frequency of various adjustments. A site can

have multiple adjustments, which is why the total number of adjustments in this table is greater than

the sample size. Also note that a single site visit may have included the randomly sampled job and its

sibling(s) and may be represented multiple times with the same savings adjustment type.

Table E-3. PY10 Midstream Lighting Verified Savings Adjustments Summary Shown in Order of Frequency

Savings Adjustment Type Number of

Adjusted Jobs Percentage of

Adjusted Jobs(1)

Facility Type 80 92%

Hours of Use 80 92%

Space Conditioning 78 90%

Coincidence Factor 74 85%

Baseline Lamp/Fixture Wattage 47 54%

Post-Install Lamp/Fixture Wattage 21 24%

Fixture Control Type 20 23%

Post-Install Lamp/Fixture Quantity 3 3%

Pre-Install Lamp/Fixture Quantity 3 3%

Total Number of Adjustments 406

(1) Percentage of adjusted jobs is calculated based on the total of 87 sample job verifications.

The following section describes the adjustments listed above.

Facility Type, Hours of Use, and Coincidence Factor

Cadmus adjusted the facility type in 92% of jobs (80 out of 87). As mentioned previously, this is because

the facility type is unknown by the ICSP. The verified facility type, in turn, determines the coincidence

factor and hours of use inputs from the IMP. Cadmus adjusted the hours of use at 92% and coincidence

factor at 85% of jobs, respectively.

Space Conditioning

Cadmus adjusted the space conditioning type at 90% of jobs (78 out of 87) as a result of a site visit or

desk audit. As mentioned previously, this is because very little is known by the ICSP about the facility

where the program lamps/fixture would be installed.

Baseline Lamp or Fixture Wattage

Cadmus adjusted the baseline wattages at 54% of jobs (47 out of 87) for the following reasons:

• In 22 jobs with LED High Bay/Low Bay products, the baseline wattages in PPL Electric Utilities’

tracking database differed from the IMP baselines. This was expected, as Cadmus referred to the

latest version of the IMP for PY10, which was approved in January 2019 after the conclusion of

Q2. This IMP is effective June 1, 2018 to May 31, 2020.


• The IMP assumes a one-for-one linear fluorescent lamp to LED lamp replacement to determine

the difference between pre-install and post-install lamp wattage.

In preparing the modified Appendix C for jobs with site visit verification, Cadmus used fixture

watts to determine the difference from pre-install to post-install wattage. Particularly for linear

tubes, wattage is tabulated by fixture not lamp. Cadmus adjusted the baseline and retrofit watts

when the IMP wattage (number of lamps x lamp wattage) was unequal to the fixture wattage.

For example, the most common linear T8 fluorescent fixture (F42ILL) is two-lamp four-foot, 59

watts. This is the source of the 29.5 watts baseline wattage for LED linear four-foot lamp in the

IMP Table 5 (59/2=29.5). However, a four-lamp four-foot fixture draws 112 watts or 28 watts

per lamp (112/4=28).

Therefore, Cadmus made verification adjustments to three-, four-, and six-lamp four-foot T8

fixture baseline wattages according to the IMP. This affected 22 jobs (nine site visits).

Cadmus did not ask about in situ fixture configuration during desk audit phone surveys, as most

respondents were not expected to provide accurate fixture configuration during phone surveys

(e.g., number of lamps per fixture). Cadmus used the default fixture configuration assumption in

the IMP lamp baseline wattage as opposed to fixture baseline wattage.

• In three jobs, the exterior fixtures were reported in PPL Electric Utilities’ tracking database as

high-bay products and vice versa. Exterior fixtures have a higher baseline wattage in the IMP

than interior high-bay fixtures.

Post-Install Lamp or Fixture Wattage

Cadmus adjusted the post-install wattages at 24% of sites (21 of 87 verifications). Cadmus adjusted the

post-install lamp or fixture wattages where the reported values differed (often slightly) from the tested

and verified wattages documented by the DesignLights Consortium. Of the 21 jobs with baseline

wattage adjustments, 13 jobs had an adjustment of less than 1 watt and the adjustment was due to the

ICSP rounding the DLC reported wattages from two decimal points to one or no decimal points.

Fixture Control Type

The Midstream IMP assumes a savings factor (SVG) of 1.44% for manual on/off switches. This refers to

the 2014 SWE Commercial and Residential Light Metering Study finding that on average 6% of

commerical lighting load is controlled by sensors, including wall-mounted sensors (6% x 24% = 1.44%).148

Post-install lighting controls were not changed as part of the retrofit in any of the projects verified. After

the phone survey or site visit verification, Cadmus adjusted the IMP default for pre-install control type

148 Statewide Evaluation Team. January 13, 2014. Pennsylvania Statewide Act 129 2014 Commercial & Residential Light Metering Study.


(consistent with post-install control type) to match the verified control type for the spaces where the

Midstream equipment was installed. In all cases, Cadmus adjusted the verified SVG as follows:

• After verifying a manual on/off switch, Cadmus adjusted the SVG factor from the 1.44% IMP

default to 0%. Due to its prevalence in almost all jobs, Cadmus did not add this adjustment to

the counts presented in Table E-3.

• At 20 jobs the adjustments to SVG were necessary to reflect pre- and post-installation automatic

lighting controls.

Post-Installation Lamp or Fixture Quantity

Cadmus revised the post-installation lamp or fixture quantities when it could not confirm installation of

the lamps sold as part of the job, the lamps were intended as spares (and therefore put in storage),

and/or the customer did not plan to install them before the end of the program year evaluation period.

Of 21,718 total lamps reported in PY10, the majority (21,075) lamps were either installed and in

operation or in-storage, soon to be installed (before the end of the program year evaluation period,

defined as August 31, 2019). Therefore, the ISR, the percentage of reported incentivized quantity across

the evaluation sample verified by Cadmus was 97%. Specifically, Cadmus found the following:

• In 83 out of 87 jobs, the ISR was 100%.

• 643 lamps could not be verified as installed or soon to be installed for four jobs at three sites:

▪ In four jobs, 485 lamps were put in storage and the customer was not planning to install the

stored lamps prior to the end of the program year evaluation period.

▪ In three of those jobs, another 158 lamps, could not be verified as installed or stored.

• The discrepancies between reported and installed lamps were small except for two jobs (at one

site). This site had placed 473 of 637 purchased lamps in storage for distribution to other

locations. During the site visit, Cadmus only verified 12 lamps were installed. Another 152 lamps

could not be verified as installed or stored. The customer was not able to confirm the timeframe

for stored equipment installation or the percentage of stored or missing equipment that would

be or was installed in PPL Electric Utilities territory. The ISRs for the two jobs at this site were

2%.

Pre-Installation Lamp or Fixture Quantity

At three jobs (one site), site-visit verification showed that four-foot four-lamp T8 fixtures were

delamped and retrofitted to four-foot two-lamp T8 LED fixtures.

Unverified Savings

Reported savings for one job in PY9 were treated as unverified. During the verification site visit in PY10,

the customer reported that all rebated equipment used on this job was returned to the distributor.

Cadmus verified that 92% of the reported quantity was returned to the distributor. The Cadmus site

inspector was not able to verify installation of the remaining quantity during the site visit.


E.2 Process Evaluation for Midstream Lighting

E.2.1 Additional Findings

This section presents additional process evaluation findings for the Midstream Lighting Program.

Participant Profile

Distributors

The following is a summary of participating distributors Cadmus interviewed:

• Five distributors reported a multistate presence, and their self-reported estimates of sales to

customers in PPL Electric Utilities’ service territory varied widely.

• Seven distributors said contractors or electricians made up the majority of their lighting sales.

• Two said sales between contractors and end users was equal.

• Two said end-user purchasers made up 75% to 95% of their sales.

• Two said energy service companies made up 25% to 50% of their sales.

• Only one distributor said the firm sold a fair amount of lighting to residential customers.

Contractors

The following describes interviewed contractors who purchased Midstream Lighting products:

• All were the primary lighting suppliers for their customers.

• Most had companies with 10 or more employees (5 of 7).

• Most served residential and commercial customers of various sizes (6 of 7).

• Most provided replacement lighting and manage maintenance repair operations (5 of 7).

• Most reported their clients keep less than 5% lighting stock in storage (4 of 7).

End Users

The following describes interviewed end users who purchased Midstream Lighting products:

• Company sizes ranged from 50 employees to 35,000 employees.

• Most managed maintenance repair operations (5 of 7).

• Most kept less than 5% of their lighting stock in storage (5 of 7).

Distributors’ Stocking and Sales Patterns

Seven interviewed distributors estimated their percentage of sales in three categories, as shown Table

E-4. These percentages are consistent with findings in PY9, where distributors reported that LEDs were

the majority of their lighting sales, specifically LEDs that were eligible for this program.


Table E-4. Distributor Sales Estimates Across Three Categories of Lighting Product Efficiency

Program-

Qualifying

Products

Non-Program-

Qualifying Efficient

Products

Standard Efficiency

Products

Multiyear Distributor 85% 15% 0%



First-Year Distributor 40% 40% 20%


First-Year Distributor 30% 30% 40%


Average 53% 20% 27%

As was the case in PY9, distributors believed the next key change in the lighting industry will be the

increased prevalence of controls, both standalone and those integrated into fixtures. These distributors

said the Midstream Lighting Program should provide discounts on these products to continue to

influence the lighting marketing.

Several distributors said they had started serving smaller customers and were seeing some customers

more frequently to do smaller jobs. They attributed this change to Midstream Lighting.

Program Influence

Distributors Interactions and Promotional Practices

Consistent with PY9, Midstream Lighting did not impact how distributors interacted with their

customers but did lead them to frequently discuss the discounts with customers. Most distributors

always or often helped their clients with their lighting purchases, both prior to participating in the

program and currently. The 10 distributors who had not changed the lighting products they promote

said they always or often tell customers about Midstream Lighting discounts when considering program-

eligible products. Most distributors said they used and were generally satisfied with the counter

marketing materials provided by the ICSP. Two distributors requested more marketing materials that

speak directly to customers, and one wanted the signage refreshed.

Contractors’ Promotional Practices and Business

When deciding to install efficient or standard lighting, all seven contractors interviewed thought

customers took these primary factors into account: the cost of the products, the return on investment in

energy savings, the application for which the lighting would be used, and the quality and longevity of the

products. Contractors said they considered the same primary factors when deciding which lighting

products to promote to their clients. Unlike PY9, no contractors identified the ease of installation and

maintenance of lighting products as a primary factor in their customers’ decision to install efficient

lighting.

One contractor recommended energy-efficient replacements for bulbs expected to be obsolete soon.

Five contractors thought energy efficiency lighting was cost-effective for their clients in every situation;


two contractors said energy-efficient upgrades were cost-effective most of the time. When

recommending high-efficiency lighting, all contractors either always (n=3) or often (n=4) incorporated

the amount of the program’s discount in the bid for their projects.

Table E-5 shows the frequency with which contractors recommend and customers request high-

efficiency lighting, particularly ENERGY STAR or DesignLights Consortium (DLC)-certified products.

Table E-5. Contractor Recommendations and Client Requests

How often contractors

recommend high-efficiency lighting

How often contractors tell

clients about the Midstream

Lighting discounts

Increase in recommendation of high-efficiency

lighting since start of Midstream

Lighting

How often clients ask for ENERGY

STAR or DLC lighting

Percentage of clients who know the

difference between ENERGY STAR or DLC

products and non-certified products

Contractors (n=7)

5 always 2 often

4 always 2 often

1 sometimes

2 yes (small increase)

5 no

5 sometimes 2 rarely

1; about 5% 1; about 25% 2; about 50% 1; about 75%

2 unsure

Midstream Lighting appears to have influenced contractors in making recommendations to their clients

and their clients’ decision to install efficient lighting, as shown in Figure E-3. Both distributors and the

discounts available through the program were rated by contractors as influential to their promotion of

qualifying lighting products, as in PY9 and PY8.149 All contractors said the program had minimal to no

influence on their sales of non-program-eligible but efficient products.

149 In PY8, six contractors said the discounts were extremely influential, while five said the distributor was extremely influential. In PY9, eight contractors said the distributor was extremely influential, and seven said the discount was extremely influential.


Figure E-3. Influences on Contractor’s and Client’s Efficient Lighting Decisions

Source: Contractor survey questions: F6. How influential, using a scale from 1, meaning no influence, to 5, meaning

extremely influential, would you say the instant discounts are in your decision to promote the ES and DesignLights Consortium

(DLC) qualified lighting products to your customers?; F7. How influential was your distributor’s recommendation in your

decision to promote the ES and DesignLights Consortium (DLC) qualified lighting products?; F13. You rated the instant discounts

as a [READ IN ANSWER TO E6] in terms of their influence on your decision to promote ES or DesignLights Consortium (DLC)

products. Using this same scale, how influential do you think the instant discounts are to your clients’ decisions to install high

efficiency lighting?; G4. How influential has PPL’s Distributor Instant Discount Program been on your sales of high efficiency

lighting that do not qualify for the instant discounts?

Similar to PY9, contractors saw a slightly positive effect on their business because of the Midstream

Lighting incentives. Five of the seven contractors interviewed thought Midstream Lighting helped

increase their revenue and sales volume, and two of these said the impact was substantial. Most

contractors estimated that only 10% to 25% of their 2018 sales were for projects that included lighting

that could qualify for Midstream Lighting discounts. But one said more than 95% of the firm’s 2018 sales

could qualify for the discounts and estimated that 30% of the firm’s total 2018 revenue was from

lighting that received a discount.

Contractors estimated that 2% to 30% of their customers still purchased standard-efficiency lighting

products, an average across all contractors of 18%.

E.2.2 Survey Approach

Survey Frequency

Cadmus conducted telephone interviews with 13 distributors and 14 purchasers in May and June 2019.

Purchasers included seven end-user purchasers and seven contractors. Additional details are in

Appendix R Survey Methodology.


Sample Attrition

Table E-6 lists total numbers of records and the outcome (final disposition) of each record.

Table E-6. Midstream Lighting Program Sample Attrition Table

Description of Call Outcomes

Number of Records

Distributors Contractors End User

Purchasers

Telephone Interview

Population (number of contact names) (1) 19 6,214 (2) 6,214 (2)

Removed: inactive customer, completed survey in past 3 months, on "opt out" list, selected for a different survey, duplicate contact, on “do not contact” list, incomplete or invalid phone number

0 5,900 5,166

Interview Sample Frame (used for interview calls) 19 314 1,048

Not attempted 0 295 1,020

Records Attempted 19 19 28

Non-working number and wrong number 0 0 1

Refusal 1 1 2

No answer/answering machine/phone busy/non-specific or specific callback scheduled

5 11 18

Partial complete (not included in interview findings analysis) 0 0 0

Total Completed Surveys 13 7 7

Telephone response rate 68% 37% 25%

(1) Number of contacts available at the time of the survey effort. (2) Contractors and End-User Purchasers started from the same population list.

Appendix F. Evaluation Detail – Custom Program PPL Electric Utilities | F-1

Evaluation Detail – Custom Program

F.1 Gross Impact Evaluation

F.1.1 Evaluation Sampling Approach

Cadmus defined projects in three strata:

• Large stratum. During the application process, projects with an expected energy savings greater

than 500,000 kWh/yr were assigned to the large stratum. Projects that were unusually

complicated or had a high level of uncertainty in the expected energy savings could be added to

this stratum, as determined by the ICSP or Cadmus. There were 26 projects in this stratum in

PY10.150 Cadmus verified savings for all of these projects.

• Small stratum. Projects with expected savings below 500,000 kWh/yr were assigned to the

small stratum. There were 44 small stratum projects reported in PY10; Cadmus verified savings

for a sample of 10 projects.

• Combined heat and power (CHP) stratum. All CHP projects were assigned to this stratum. There

was one CHP project with reported savings in PY10. Cadmus verified savings for that project.

Cadmus did not identify a “high interest” substratum in PY10, but it may be added in future program

years. A high-interest substratum would include projects where equipment or systems exhibit high

uncertainty in either system/equipment operation, calculated savings, or both. This substratum would

include new or emerging technologies under consideration by PPL Electric Utilities and the ICSP for new

offerings or approaches.

The agricultural improvement type was new in PY10, with two projects reported, representing less than

1% of reported savings. One of the two projects was randomly selected into the small stratum

F.1.2 Ex Post Verified Savings Methodology

Cadmus evaluated all sampled projects, verifying savings at a high level of rigor, using approaches

described in the International Performance Measurement and Verification Protocol (IPMVP). A

discussion of the approach, by stratum, follows.

Large Stratum

The ICSP either calculated the initial savings or used contractor estimated savings (called reserved

savings) to determine which projects entered into the large stratum.151 The ICSP informed Cadmus of

these projects during the application process. Calculation methodologies and verification approaches

vary by project. Cadmus prepared the site-specific measurement and verification plan (SSMVP), typically

150 The large stratum population total is 28, but it includes two incentive adjustments without claimed savings that were not included in the sample frame for PY10.

151 Reserved savings are based on early customer or contractor estimates of baseline and proposed equipment energy use and do not necessarily represent the reported or verified project savings.


in coordination with the ICSP, and conducted pre-installation inspections to gather baseline data for all

large stratum projects, except new construction, for which there was no existing condition.

Cadmus conducted post-installation site visits and other customer outreach to verify installation and

gather additional data to verify energy savings. For some large projects, Cadmus installed data logging

equipment, collected data from a customer control system through trends or spot readings, or gathered

equipment and operating information from customer interviews.

Cadmus verified savings for all large and CHP strata projects prior to the ICSP reporting project savings.

Small Stratum

At the end of each quarter in PY10, Cadmus randomly selected projects to include in the small stratum.

Cadmus did not conduct pre-installation inspections because small stratum projects cannot be sampled

until after equipment is installed and the incentive is paid. Cadmus prepared the SSMVP for each project

and then conducted post-installation inspections and verified savings.

Cadmus calculated the realization rate for the selected sample as the ratio of ex post verified gross

savings to ex ante savings and then applied this realization rate to the entire small stratum population.

CHP Stratum

All CHP projects with claimed savings in PY10 were included in the evaluation sample. Cadmus prepared

the SSMVP in coordination with the ICSP, then conducted a post-installation site visit to verify

equipment operated as designed. Data were collected for three to six months to determine CHP-

generated electricity, parasitic loads, useful heat recovered from the CHP, and net gas usage (CHP gas

consumption less gas usage offset by heat recovery). A regression analysis was conducted comparing

CHP electric generation, useful heat recovered, and natural gas usage to any related independent

variables (e.g., outside air temperature) and then annualized using a year of typical data (e.g., TMY3

weather data, average annual production) to determine first year project savings.

F.1.3 Realization Rate Findings

Cadmus found various reasons for the differences between ex ante and ex post savings, detailed below.

Large stratum. There is no realization rate discrepancy in the large stratum. Projects in this stratum are

evaluated prior to being reported, so ex ante savings are equal to ex post savings.

Small stratum. For projects in the small stratum, the ICSP’s and Cadmus’ savings methodologies differed

depending on the information available, customer data trending capabilities, the ratio of estimated

savings to overall customer usage, and Cadmus’ ability to deploy logging equipment. Cadmus noted

these sources for discrepancies in realization rates in small stratum projects:

• For five projects, Cadmus collected additional post-installation metering or utility data. The

expanded post-installation data sets resulted in differences between the reported and verified

savings:


▪ For three projects, the savings were reduced between 6% and 46%.

▪ For two projects, the savings were increased between 4% and 20%.

• For three projects the ICSP used deemed hours and savings factors, similar to what is used in a

prescriptive program. The evaluated savings for these projects used a custom workbook analysis

and metering data.

• For a production equipment product (plastic extruder), the evaluation dataset was larger than

the ICSP’s. The project involved a supply-side improvement, though the demand differed

between the baseline and post periods. To normalize savings, Cadmus applied the baseline and

post equipment regressions to the combined set of pre- and post-period data, (rather than only

applying the regressions to one period as in the ICSP’s ex ante analysis.)

• The baseline operating parameters were updated for one refrigeration project that used an

industry standard calculator. The scope of the project did not allow for an effective metering

solution, so the calculator input parameters were verified. The ICSP did not provide sources for

baseline inputs, so the evaluation relied on customer interviews and baseline photo reviews

(photos provided in project documentation) to verify the baseline inputs.

CHP stratum. Data entry errors in reporting demand savings for one project caused differences between

reported and verified demand savings. No errors were found with usage (kWh) savings.

F.2 Net Impact Evaluation

F.2.1 Net-to-Gross Ratio Findings

Cadmus summed the intention and influence components of the net savings algorithm to estimate the

free ridership average, weighted by ex post gross program savings. Table F-1 summarizes the intention,

influence, and free ridership score.

Table F-1. PY10 Custom Program Intention, Influence, and Free Ridership Scores

n Intention Score Influence Score Free Ridership Score

21 31% 4% 35%

For the four largest projects (all large stratum) of the survey respondents in PY10, the savings-weighted

free ridership score was 28%. These four projects represented 76% of the analysis sample’s verified

savings,152 and they accounted for six percentage points of the program-level free ridership estimate of

35%. Table F-2 lists the sector for the four projects with the largest verified savings.

152 The four largest projects in the analysis sample represented 17% of the verified savings for the Custom Program population.


Table F-2. PY10 Custom Program Free Ridership for Four Top Saving Projects

Sector/Stratum of Four Largest Projects included in Free Ridership

Surveys

Verified kWh/yr Savings

Percentage of Analysis Sample Verified Savings

Percentage of Program

Population Verified Savings

Free Ridership

GNE/Large 5,589,585 32% 9% 25%

Large C&I/Large 2,797,796 16% 4% 25%

Large C&I/Large 2,517,035 15% 4% 62.5%

Large C&I /Large 2,203,236 13% 3% 0%

Total (1) 13,107,652 76% 21% 28%(2)

(1) Total may not match due to rounding. (2) Weighted by verified kWh/yr savings. Relative precision at 85% confidence is 25%.

F.3 Process Evaluation

F.3.1 Additional Findings

This section presents additional process evaluation findings for the Custom Program.

Participant Profile

This section provides a profile of all customers who participated in the Custom Program and summarizes

the firmographics of survey respondents. Table F-3 shows the sectors represented in the survey

population and in the full participant population.

Table F-3. PY10 Custom Program Sector Breakdown

Sector Total Population

(n=72) (1) Survey Respondents

(n=21)

Large 38% 48%

Small 40% 19%

GNE 21% 29%

Residential 1% 5% (1) PPL Electric Utilities' tracking database includes 74 unique projects in PY10 but two of these are incentive adjustments for projects reported in a previous program year. The incentive adjustments are not included in the total population in this table. Source: PPL Electric Utilities’ tracking database; may not total 100% due to rounding.

More than three-quarters of the survey respondents (81%; n=21) said they had previously participated

in the Custom component of the Non-Residential Energy Efficiency Program before PY10.

Twelve of 16 survey respondents said their facilities were more than 100,000 square feet. Thirteen of 20

respondents said their facility had more than 100 employees.

Table F-4 shows the types of facilities participating in the Custom Program and facilities of survey

respondents. The majority of the total population and survey respondents were from the manufacturing

segment.


Table F-4. PY10 Facility Types of Custom Program Participants and Survey Respondents

Facility Use Total Population (n=72) (1) Survey Respondents (n=21) (2)

Manufacturing 42% 48%

Education 7% 14%

Grocery – supermarket or convenience store 6% 0%

Hospital or healthcare 6% 5%

Lodging 1% 0%

Warehouse 4% 10%

Agriculture 1% 10%

Dining 1% 0%

Transportation 0% 5%

Other 32% 10%(3) (1) Source: PPL Electric Utilities’ tracking database, based on unique bill account numbers. PPL Electric Utilities' tracking database includes 74 unique projects in PY10 but two of these are incentive adjustments for projects reported in a previous program year. The incentive adjustments are not included in the total population in this table. (2) Source: Survey question, “What is the primary use of your facility?”; may not total 100% because of rounding. (3) Other responses included a Government facility and a sewer facility.

F.3.2 Survey Approach

Survey Frequency

Cadmus conducted online and telephone surveys, reaching out to all PY10 participants in March 2019

(Q1 and Q2 participants) and in July 2019 (Q3 and Q4 participants). Where possible, PPL Electric Utilities

provided KAM outreach to increase response rates. The cleaning and survey sample preparation process

reduced the available sample from 74 to 38. Additional information is found in Appendix R Survey

Methodology.

Sample Attrition

Table F-5 lists the total records used for surveys and the outcome (final disposition) of each record.


Table F-5. PY10 Custom Program Participant Survey Sample Attrition Table


Online

Population 74

Removed: duplicate, inactive customer, completed survey in past 3 months, on opt out list, duplicate contact, on do not contact list, incomplete or invalid email address, no savings in PY10 (incentive adjustment)

36

Survey Sample Frame (email invitations sent) 38

Email was returned (bounce back), did not respond, opted out, or did not complete survey 36

Ineligible: PPL Electric Utilities employee/cannot confirm project location 0

Completed Surveys 2


Telephone

Population 74

Removed: duplicate, inactive customer, completed survey in past 3 months, on opt out list, duplicate contact, on do not contact list, incomplete or invalid phone number, no savings in PY10 (incentive adjustment)

36

Completed online survey 2

Survey Sample Frame (used for telephone survey calls) 36

Not attempted 0

Records Attempted 36

Not reached: no answer, answering machine, phone busy, refused 16

Ineligible: cannot confirm equipment/not aware of participation, employment, ESCO 0




Total Completed Surveys (total for all modes) 21

Overall Response Rate (for both modes) 55%

Appendix G. Evaluation Detail – Continuous Energy Improvement Program PPL Electric Utilities | G-1

Evaluation Detail – Continuous Energy Improvement

Program

G.1 Gross Impact Evaluation

G.1.1 Methodology

Background

Cadmus estimated the energy and demand savings at the 18 schools that participated in the CEI

Program. These 18 schools were from four districts, as listed in Table G-1. All schools started

participating in the program in February 2017, and Cadmus evaluated their second year of savings.

Table G-1. Evaluated Schools and Districts

District School District School

District 1

School A

District 3

School K

School L

School B School M

School C School N

School D School O

School E School P

District 2

School F

District 4

School Q School G

School H

School I School R

School J

Cadmus developed savings estimates using facility-level energy consumption data measured from

approximately one year before the schools participated in this program (the baseline period) through

their second year of participation (the performance period). This approach empirically quantifies the

impacts of the program and is consistent with the International Performance Measurement and

Verification Protocol (IPMVP) Option C.153

Datasets

Participant Documentation

PPL Electric Utilities and the ICSP provided participant documentation. The documentation contained

information on the participating schools, including their names, addresses, account numbers,

153 International Performance Measurement and Verification Committee. International Performance Measurement and Verification Protocol: Concepts and Options for Determining Energy and Water Savings. May 2016. Available online: www.evo-world.org.

http://www.evo-world.org/


participation dates, school calendars, capital projects installed, and regression model specifications and

resulting CEI savings.

Advanced Metering Infrastructure (AMI) Data

PPL Electric Utilities provided hourly interval electricity consumption data (referred to as AMI data) for

each account number at the 16 schools from approximately January 2016 through March 2019. In some

cases, individual schools had multiple account numbers that Cadmus merged at the facility level. These

data were also split into baseline and performance periods using the dates of each school’s

participation, all of which began in February 2017.

Local Climatological Data

Cadmus correlated each school’s energy consumption with weather data, including outdoor air

temperature and relative humidity. These data are available through the National Oceanic and

Atmospheric Administration (NOAA) at weather stations across the country. Cadmus used the school

addresses provided in the PPL Electric Utilities’ tracking database to locate nearby weather stations and

download datasets of hourly weather observations during the period concurrent with the AMI data.

Data Review

Cadmus visually inspected each set of time series data used in the analysis of the 18 schools. Where

data were missing, Cadmus worked with PPL Electric Utilities to obtain a complete year of data for the

baseline and performance periods. None of the AMI data contained values that were negative,

exceedingly high, or otherwise appeared unreliable.

Dataset Preparation

Cadmus prepared several datasets for each school prior to estimating savings. Each dataset contained

electricity consumption, temperature, relative humidity (RH), and class schedule data, but at varying

time intervals. These intervals were either hourly, daily, or monthly, and different sets of independent

variables were available or calculated for each interval, as shown in Table G-2. Cadmus used these

datasets to test how savings and uncertainty varied with different modeling techniques and inputs.

Table G-2. Independent Variables by Sampling Interval

Sampling Interval

Available Independent Variables

Hourly Temperature, RH, Time of Day, Day of Week, Class (in/out)

Daily HDDs, CDDs, Average RH, Day of Week, Class (in/out)

Monthly HDDs, CDDs, Class (total class days in month)

Datasets were also split into baseline and performance periods. At this step, Cadmus removed the entire

month of February 2017 to standardize the start dates. The performance period of the program’s

second year for all schools was March 1, 2018, through February 28, 2019. This was consistent with the

ICSP’s approach.


Modeling

Cadmus used statistical modeling to estimate energy and demand savings for the CEI Program. These

techniques empirically quantify savings, by regressing baseline period energy consumption as a

response to local meteorological and temporal variables, and predict what a school’s energy

consumption would have been during the performance period had they not participated in the CEI

Program. The model fit to the baseline period data is referred to as the baseline model.

Cadmus tested several baseline models for each school to understand the tradeoffs in modeling data

sampled at varying intervals and in applying different modeling techniques. Cadmus used the AMI data

to attempt to improve the precision of the reported savings over models using monthly data.

Developing Models with AMI Data

Monthly billing data are often used in estimating energy savings, but when higher frequency data are

available it may be possible to improve the precision of the results by applying additional inputs and

alternative modeling techniques. Cadmus tested two modeling techniques, multiple linear regression

and random-forest regression, on hourly, daily, and monthly datasets for each school. It is essential that

the process of comparing these combinations of datasets and techniques is objective in the selection of

independent variables and hyperparameters.

An additional step for monthly and daily datasets is choosing the base temperatures to calculate heating

degree days (HDDs) and cooling degree days (CDDs). The fit of a baseline model depends on this choice,

so Cadmus tested a range of base temperatures by fitting a linear model to each combination for HDD

and CDD base temperatures and selected the pair yielding the highest r-squared. Figure G-1 illustrates

this process for School G, and although r-squared values do not fluctuate much with HDD and CDD base

temperatures, this procedure is more objective than using engineering judgment.

Figure G-1. Optimizing HDD and CDD Base Temperatures for School G

Next, Cadmus selected the independent variables for each baseline model. It is best practice to exclude

variables that do not correlate with the energy consumption, correlate highly with other variables, or do

not predict well out of sample. Cadmus first plotted all candidate variables against energy consumption

to confirm there was a correlation, then recursively tested combinations of independent variables using


cross-validation. This process determined the final inputs for each school’s baseline model. Additionally,

if any capital equipment improvements were installed during the baseline period, Cadmus included an

indicator variable in the model to signify the equipment installation date.

Many machine learning techniques, including random-forest regression, also allow for specification of

hyperparameters, and the reliability of out-of-sample predictions can vary drastically with the choice of

these inputs. An important hyperparameter for models using branching, tree-like structures, is the

number of observations in each end node or leaf. When leaf sizes are very small, models can appear to

fit extremely well but fail to predict with similar accuracy. Cadmus used grid-search cross-validation over

a range of hyperparameters to avoid over-fitting baseline models.

Cadmus scored baseline models using several metrics, including r-squared, coefficient of variation root-

mean-square error, and normalized mean bias error, and bounded total estimated performance period

consumption using prediction intervals. These metrics informed Cadmus’ final choice in fitting baseline

models using daily consumption data and random-forest regression.

The whole-facility savings were then determined by subtracting the actual consumption during the

performance period from the predicted baseline consumption.

Six schools participated in other PPL Electric Utilities’ rebate programs in PY9 and PY10 and installed

energy-efficient equipment during their participation in the CEI Program. Cadmus subtracted the

prorated ex post savings for these equipment upgrades from the whole-facility savings to determine the

savings for the CEI Program. The prorated ex post savings are included in the verified savings for the

Efficient Equipment Program.

The whole-facility, capital project, and CEI Program energy savings are shown in Table G-3. The capital

savings column represents cumulative midstream savings post-CEI implementation and includes PY9.

Table G-3. Evaluated Energy Savings by School

School Whole-Facility Savings (kWh)

Capital Project Savings (kWh)

CEI Program Savings (kWh)

85% Confidence Lower Bound

(kWh)

85% Confidence Upper Bound

(kWh)

Savings as a Percentage of

Baseline Consumption

School A -29,307 0 -29,307 -32,898 -25,715 -4.80%

School B 49,444 0 49,444 41,106 57,781 6.90%

School C 6,346 0 6,346 -12,787 25,479 0.25%

School D 119,537 0 119,537 109,157 129,918 6.73%

School E 32,762 0 32,762 24,181 41,342 4.11%

School F 35,217 0 35,217 22,059 48,375 4.03%

School G -11,030 0 -11,030 -18,209 -3,851 -2.22%

School H 39,413 0 39,413 31,612 47,214 9.05%

School I 3,053 0 3,053 -3,353 9,459 0.66%

School J -8,522 0 -8,522 -19,747 2,704 -1.30%

School K 76,598 6,289 70,309 65,823 74,794 16.42%

School L 72,417 5,078 67,339 60,744 73,934 13.50%


School Whole-Facility Savings (kWh)

Capital Project Savings (kWh)

CEI Program Savings (kWh)

85% Confidence Lower Bound

(kWh)

85% Confidence Upper Bound

(kWh)

Savings as a Percentage of

Baseline Consumption

School M 1,463 0 1,463 -5,735 8,661 0.15%

School N 21,766 35,724 -13,958 -18,005 -9,911 -5.52%

School O 474,554 76,361 398,193 350,365 446,022 6.95%

School P 27,755 0 27,755 21,643 33,868 7.58%

School Q -162,065 34,422 -196,487 -220,603 -172,371 -8.85%

School R -7,136 35,283 -42,419 -55,038 -29,799 -3.67%

Total 742,265 193,157 549,108 2.62%

Cadmus also used these models to estimate demand savings, by first fitting them to peak period hourly

data during the baseline period,154 then predicting hourly consumption during the performance period.

Cadmus then calculated whole-facility demand savings by averaging the hourly savings during the peak

period. To determine the CEI Program demand savings, Cadmus subtracted ex post capital project

demand savings from the whole-facility demand savings.

The whole-facility, capital project, and the CEI Program demand savings are shown in Table G-4.

Table G-4. Evaluated Average Demand Savings by School (kW/yr)

School Whole-Facility

Savings

Capital Project

Savings CEI Savings

Savings Lower

Bound

Savings Upper

Bound

CEI Savings

(%)

School A 8.41 0.00 8.41 7.14 9.68 14.88%

School B 18.91 0.00 18.91 17.14 20.69 29.13%

School C 12.89 0.00 12.89 8.61 17.16 3.71%

School D 43.12 0.00 43.12 40.59 45.65 24.67%

School E 9.66 0.00 9.66 7.61 11.71 8.17%

School F 3.53 0.00 3.53 1.49 5.58 4.95%

School G 1.43 0.00 1.43 0.46 2.40 4.84%

School H 10.47 0.00 10.47 9.23 11.71 29.57%

School I -2.17 0.00 -2.17 -3.24 -1.10 -8.77%

School J -3.48 0.00 -3.48 -5.00 -1.96 -7.48%

School K 14.24 0.12 14.12 13.09 15.15 24.48%

School L 22.05 0.85 21.20 20.21 22.19 27.66%

School M 2.39 0.00 2.39 0.73 4.06 2.40%

School N 8.06 6.06 2.00 0.98 3.02 4.22%

School O 174.04 0.26 173.78 159.13 188.43 22.29%

School P 12.24 0.00 12.24 11.01 13.48 19.31%

School Q -91.96 6.09 -98.05 -101.34 -94.76 -45.65%

School R -53.23 5.98 -59.21 -61.64 -56.79 -42.14%

Total 190.60 19.36 171.24 6.99%

154 Peak period is defined as June through August excluding weekends and holidays, 2:00 p.m. to 6:00 p.m., according to the 2016 PA TRM in Table 1-3.


Savings Realization Rate Methodology

Cadmus calculated the program realization rate by dividing the ex post evaluated savings by the ex ante

reported savings. Extrapolation of the realization rate to the population is not necessary for the CEI

Program since the realization rate is based on a census of all schools. No realization rate was calculated

for demand savings because there were no ex ante savings reported.

G.1.2 Database Review Findings

Cadmus reviewed the savings reported in the PPL Electric Utilities tracking database to determine

whether reported savings matched the savings documented in the ICSP’s M&V Reports and found that

all reported savings aligned.

G.2 Net Impact Evaluation

G.2.1 Net-to-Gross Ratio Methodology

Because the CEI Program is demanding and focuses on O&M and energy-saving behavior activities, very

little free ridership is expected. It can also be difficult to measure, and the billing analysis captures both

free ridership and spillover. It would be time-intensive to ask energy managers the intention and

influence questions for every activity implemented at every school and impossible to attribute savings to

individual behavior and O&M activities. Therefore, Cadmus focused interview questions on the

program’s overall influence on capital, O&M, and behavioral projects implemented during PY10 to

provide qualitative insight.

G.2.2 Net-to-Gross Ratio Sampling

The sampling strategy is presented in Table G-5.

Table G-5. PY10 Continuous Energy Improvement Sampling Strategy for NTG Research

Stratum Stratum

Boundaries

Population

Size (1)

Assumed

Cv or

Proportion

in Sample

Design

Assumed

Levels of

Confidence

& Precision

Target

Sample

size

Records

Selected

for Sample

Frame

Achieved

Sample

Size

Percent of

Sample

Frame

Contacted

to Achieve

Sample (2)

School

District Pilot

Schools

Pilot schools 4 N/A N/A 4 4 4 100%

(1) Represents number of energy managers. (2) Sample frame is a list of contacts that have a chance to be selected into the sample. Percent contacted means the percentage of

the sample frame called to complete interviews.

G.2.3 Program Influence Findings

Cadmus asked energy managers to rate the influence of the CEI Program on the decision to implement

O&M, behavioral, and capital projects, using a scale of 1 (no influence) to 5 (extremely influential). Table

G7 presents these results.


Cadmus also asked energy managers which of these types of activities they would have implemented in

absence of the program. Lastly, Cadmus asked energy managers how much of the energy savings that

the ICSP estimated were achieved as of December 2018 were a direct result of the district’s participation

in the Continuous Energy Improvement Program. Table G-7 presents these results and shows that the

CEI Program was highly influential on all three types of projects, and at least hastened the

implementation of projects that would have been completed in absence of the program.

Table G-6. PY10 Continuous Energy Improvement NTG Findings

District

Program Influence (1) 1 (no influence) to 5 (extremely influential) Projects completed during PY10 that would have

been completed in absence of program Capital Projects O&M Projects

Behavioral Activities

1 4 5 4 Some lighting and a proportion of commissioning

but would have been slower without CEI.

2 4 N/A 2 or 3(3) Lighting, but sooner because of CEI; chiller

replacement would have been 1-2 years later

3 4(2) 4 4 None

4 N/A 5 5 None (1) The question was “Please rate how influential the CEI program was overall on the completion of the following types of projects at all the schools in the district over the past year using a scale from 1, meaning no influence, to 5, meaning the CEI program was extremely influential.” (2) This district’s energy manager rated their plans to implement capital projects as a 4, but said they had not actually implemented those projects. (3) This district’s energy manager rated the program’s influence in behavioral activities a 2 or 3 in PY10, but said they would not have done any without the program, then explained that these activities were based on what they had implemented via the program in PY9.

Appendix H. Evaluation Detail – Efficient Lighting Program PPL Electric Utilities | H-1

Evaluation Detail – Efficient Lighting Program

H.1 Gross Impact Evaluation

H.1.1 Baseline Adjustments

Table H-1 lists the baseline adjustments from Section 10.2.2.1 ENERGY STAR Verification.

Cadmus also used the QPL’s efficient wattage ratings in its verified savings calculations. Section 10.2.2.1

Tracking Data Review provides more detail about comparisons between data tracking in PPL Electric

Utilities’ tracking database and the QPL.

Table H-1. PY10 Baseline Wattage Adjustments by PA TRM Bulb Category

TRM Bulb Category Lumens Reported

Baseline Wattage Evaluated

Baseline Wattage Quantity of Bulbs

Adjusted

3-Way (Exempt) 1,400 53 75 2,540

3-Way (Exempt) 1,500 53 75 1,058

3-Way (Exempt) 1,420 53 75 2,441

3-Way (Exempt) 1,200 53 75 5,286

3-Way (Exempt) 1,400 72 100 275

3-Way (Exempt) 1,600 72 100 135

3-Way (Exempt) 2,150 72 150 2,577

Candelabra/Decorative (1) 350 29 40 26,939






Globe 350 29 40 36,412

Globe 450 29 40 15,001

Globe 470 29 40 2,456

Reflector 650 50 65 25,328

Reflector 990 75 40 3,054

Reflector 940 75 40 5,712

Reflector 980 75 40 492

Reflector 750 75 50 430

Reflector 800 75 50 75

Reflector 850 75 55 825

Reflector 1,400 90 55 84

Reflector 1,750 120 45 718

Reflector 1,500 150 55 3,525

Total 281,433

(1) Medium (E26) base.


H.2 Process Evaluation

H.2.1 Survey Approach

Survey Frequency

Cadmus conducted one phone survey with residential customers in May 2019, and one phone survey

with small business customers in May 2019.

Sample Attrition

Table H-2 and Table H-3 list the total number of records submitted to the survey subcontractor and the

outcome (final disposition) of each record.

Table H-2. Residential General Population Phone Survey Attrition

Description of Outcomes Number of

Records

Population 1,226,766

Removed: inactive customer, completed survey in past 3 months, on "do not contact" list, selected for a different survey, duplicate contact

278,948

Incomplete or invalid phone number or email 28,909

Survey Sample Frame 918,908

Not attempted 907,991

Records Attempted 10,917

Non-working, wrong number, business, language barrier 1,809

Not reached: No answer, answering machine, phone busy, refused, terminated, non-specific or specific callback scheduled

8,758

Screened out: PPL Electric Utilities or market research employee, cannot confirm equipment/not aware of participation

50


Response rate 3%


Table H-3. Small Business General Population Phone Survey Attrition


Records

Population 183,289

Removed: inactive customer, completed survey in past 3 months, on "do not contact" list, selected for a different survey, duplicate contact

74,265

Incomplete or invalid phone number or email 15,009

Survey Sample Frame 94,015

Not attempted 76,838


Non-working, wrong number, business, language barrier 2,609


14,074


96


Telephone Response rate 2%

Appendix I. Home Energy Education Program PPL Electric Utilities | I-1

Evaluation Detail – Home Energy Education Program

I.1 Process Evaluation

This section includes additional process evaluation findings and details on the survey approach.

I.1.1 Additional Findings

Treatment Group Customer Profile

The customer survey also collected demographic details about the treatment group customers. The

majority of treatment group customers (home energy report recipients) had the following

characteristics:

• Lived in a single-family detached residence (75%, nw=408)

• Had an average household size of 2.7 people (nw=402)

• Averaged 59.6 years of age (nw=387)

• Had completed some college education or more (68%, nw=406)

• Had an annual household income of $60,000 or greater (57%, nw=356)

I.1.2 Survey Approach

Survey Frequency

Cadmus conducted a telephone and online survey with treatment group customers once during PY10 in

late April 2019. Additional details are in Appendix R Survey Methodology.

Sample Attrition

Table I-1 lists total number of records submitted and the outcome (final disposition) of each record for

the online and telephone surveys.

Table I-1. Treatment Group Customer Survey Sample Attrition Table

Description of Outcomes Number of Records

Population (Number of Unique Customers) (1) 250,989

Removed: inactive customer, completed survey in past 3 months, on "opt out" list, selected for a different survey, duplicate contact, on “do not contact” list, missing or invalid email address, missing or invalid phone number

152,278

Online Survey

Selected for survey sample frame (sent to subcontractor for online survey emails) (2) 69,292

Not attempted (3) 7,187


Email was returned (bounce back or failed), did not respond, opted-out, or did not complete survey 6,347

PPL Electric Utilities or market research employee 95

Cannot confirm awareness of home energy reports 76

Completes 197


Appendix I. Home Energy Education Program PPL Electric Utilities | I-2

Description of Outcomes Number of Records

Telephone Survey

Selected for survey sample frame (sent to subcontractor for telephone survey calls) (2) 29,419

Not attempted (3) 1,683


Non-working number, wrong number, or business 4,223

No answer, answering machine, phone busy, callback 13,715

Language barrier 40

PPL Electric Utilities or market research employee 174

Cannot confirm awareness of home energy reports 129

Refusal or terminated survey 426

Completes 219


Total Completes (all modes) 416

(1) Number of records available in ICSP’s database at the time of the final survey effort. (2) Not selected for sample because there were more records than needed for the telephone survey. (3) Selected for sample but target was reached before attempted.

Survey Data Analysis

To analyze the survey data, Cadmus compiled frequency outputs, coded open-end survey responses,

and conducted statistical tests. To determine whether survey results significantly differed between

waves and program years, Cadmus ran t-tests for differences in proportions and means set at the 5%

(p≤0.05) and 10% (p≤0.10) significance levels.

Cadmus applied statistical weights at the wave level to reflect actual program population proportions.

Survey results reported at the program level (when all waves are combined) were weighted by wave and

are indicated by the notation nw. Survey results showing comparisons between waves were not

weighted. Table I-2 shows the statistical weights applied to the treatment group survey responses.

Weighted survey data are indicated by the notation nw in the report and figures.

Table I-2. Statistical Weights for Overall Treatment Group Customer Survey Data

Wave Stratum Population

Count (1) Proportion of

Population

Survey Sample

Achieved

Proportion of Total Survey

Sample

Statistical Weight (2)

Phase I Legacy Waves 83,605 33.3% 103 24.8% 1.345

Phase II Expansion Wave 46,578 18.6% 101 24.3% 0.764

Phase III Expansion Wave 30,584 12.2% 103 24.8% 0.492

Phase II Low-Income Wave 1 69,354 27.6% 89 21.4% 1.292

Phase II Low-Income Wave 2 20,868 8.3% 20 4.8% 1.729

Total 250,989 100.0% 416 100% N/A (1) The total number of customers in the treatment group at the time of the survey activity. These numbers may not match those reported in the impact analysis sections of this report due to the timing of the survey activity. (2) The statistical weight is calculated by dividing the proportion of population by the proportion of total survey sample.

Appendix J. Evaluation Detail – Winter Relief Assistance Program PPL Electric Utilities | J-1

Evaluation Detail – Winter Relief Assistance Program

J.1 Gross Impact Evaluation

J.1.1 Methodology


The verification sample for the Winter Relief Assistance Program (WRAP) was designed to meet

requirements of 85% confidence with ±15% precision. To allow for a detailed examination of savings in

each stratum, Cadmus organized the population into five strata. Cadmus sampled the population by

project number instead of by an account number because master-metered multifamily jobs are tied to a

single account number. Table J-1 shows the sampling strategy for all of PY10.

Table J-1. PY10 Winter Relief Assistance Program Sampling Strategy

Stratum Number of

Jobs

Target Levels of Confidence & Precision (1)

Assumed Cv (2)

Target Sample

Size

Achieved Sample

Size

Impact Evaluation

Activity

Baseload 6,671

85/15 0.5

24 24

Records Review and Engineering

Analysis

Low-Cost 4,666 24 24

Full-Cost (3) 1 N/A N/A


1,621 24 24


2,267 (4) 24 24

Program Total (5) 15,226 96 96 (1) By setting the target confidence and precision to 85/15 for records review, sample sizes were sufficient to estimate energy savings for the program with confidence and precision of 85/15. (2) The PY9 evaluation found that the coefficient of variation for all strata was around 0.3, which indicates Cadmus could sample fewer jobs and still reach precision targets. To be conservative, Cadmus uses a minimum Cv of 0.5. (3) Since there was only one full-cost job in PY10, Cadmus applied the full-cost stratum’s realization rate to the PY10 job. (4) Fifty-three master-metered multifamily buildings that participated in WRAP in PY10 have 2,267 tenant units. Cadmus performed home audit records review and engineering analysis for individual tenant units in master-metered multifamily buildings. Therefore 2,267 jobs were added to the total number of jobs in the sample.


The ICSP inspected about 5% of all jobs associated with program participation to ensure project quality

and to verify that products were installed as documented. Cadmus did not use the ICSP’s verification

data to adjust the ISR for ex post verified savings because the data uploaded into the PPL Electric

Utilities’ tracking database contained the final savings after the ICSP took any remedial actions.


Cadmus calculated ex post verified savings through an engineering analysis, a review of audit records,

and two participant surveys. For each stratum, Cadmus calculated the realization rate as the ratio of

ex post verified gross savings to ex ante savings then applied this realization rate for the selected sample

to its respective stratum population.

The following sections provide an overview of the products and services offered in each of WRAP’s five

strata and the PA TRM protocols used to determine verified savings.


Job Type Definitions and Verification References

Baseload Job Type

Baseload jobs require no additional qualifications beyond the general WRAP income-eligibility

requirements. However, baseload customers generally have non-electric heating and a non-electric

water heater.155 Table J-2 shows the energy-saving items in the baseload stratum and the PA TRM

entries Cadmus used to determine verified energy savings. Customers are eligible for all items offered by

the job type, but most customers do not receive all of these items.156

Table J-2. PY10 Baseload Items for Winter Relief Assistance Program

Items Offered PA TRM References

LED Nightlight LED Nightlight - Section 2.1.4

ENERGY STAR LED Lighting ENERGY STAR Lighting - Section 2.1.1

Tier 2 Advanced Power Strips Smart Strip Plug Outlets - Section 2.5.3

Energy Education

Programmable Thermostats – Section 2.2.8 Water Heater Temperature Setback – Section 2.3.6 Low Flow Showerheads – Section 2.3.9 WRAP Participant Survey

Refrigerator Recycle with Replace

Final IMP - EDC Direct Install Refrigerator / Freezer Recycling with Replacement

Refrigerator Recycle without Replace

Freezer Recycle with Replace

Freezer Recycle without Replace

Furnace Whistle (1) Furnace Whistle – Section 2.2.7

(1) Cooling only; a furnace whistle with electric heating is a full-cost item.

Low-Cost Job Type

Homes with electrically heated water qualify for low-cost jobs. Low-cost jobs are eligible for the items in

Table J-3 and all items offered to baseload job types.

Table J-3. PY10 Low-Cost Items for Winter Relief Assistance Program


Low-Flow Faucet Aerator Low-Flow Faucet Aerators – Section 2.3.8

Low-Flow Showerhead Low-Flow Showerheads – Section 2.3.9

Water Heater Temperature Setback Water Heater Temperature Setback – Section 2.3.6

Water Heater Pipe Insulation Water Heater Pipe Insulation – Section 2.3.7

Water Heater Tank Wrap Water Heater Tank Wrap – Section 2.3.5

Thermostatic Restriction Valve Thermostatic Shower Restriction Valve – Section 2.3.10

Heat Pump Water Heater Heat Pump Water Heater – Section 2.3.1

155 If a customer had an electric water heater but refused water heater products, the customer was categorized as a baseload customer.

156 Customers do not receive all items for a variety of reasons. For example, customers refused some items or were not eligible (the customer already had the item in place, their freezer did not need to be recycled, etc.).


Full-Cost Job Type

Homes with electric space heat qualify for full-cost jobs, which include all baseload and low-cost items.

Table J-4 shows the full-cost eligible jobs.

Table J-4. PY10 Full-Cost Items for Winter Relief Assistance Program


Air Sealing Residential Air Sealing – Section 2.6.6

Attic/Wall Insulation Ceiling / Attic and Wall Insulation – Section 2.6.1

Residential Advanced Smart Thermostat Residential Thermostat IMP, finalized February 26, 2018

Residential Programmable Thermostat

Master-Metered Multifamily Buildings

Master-metered multifamily buildings were eligible for baseload, low-cost, and full-cost products.

However, in PY10, all buildings received only baseload or low-cost products. (See products and services

listed in Table J-2 and Table J-3.)

Manufactured Home Initiative

Manufactured homes with electrically heated water qualified for low-cost jobs. If the home did not have

electrically heated water, it qualified for a baseload job. In addition, some manufactured homes were

eligible for the minor air sealing improvements—door caddies, door corner pads, closed cell foam

weatherstripping, and window kits—based on the auditor’s recommendations.157 The exact combination

of products delivered, along with minor air sealing, depended on the conditions of the individual home.

Cadmus evaluated these savings using the weatherstripping IMP.158

Ex Post Savings Calculation Methodology for Energy Education

Cadmus selected three behavioral recommendations—adjusting thermostats, washing clothes in cold

water, and taking shorter or fewer showers—that reasonably corresponded to energy-saving activities in

the PA TRM.

Adjusting Thermostats

Cadmus assumed that participants who adjusted their thermostats saved energy similar to savings from

a programmable thermostat and applied the PA TRM’s algorithms accordingly.159

157 Usually these homes had electric heat. All homes had either cooling, electric heat, or both.

158 Pennsylvania Public Utility Commission. Weather Stripping, Caulking and Outlet Gaskets IMP. June 1, 2017.

159 Section 2.2.8 of the PA TRM.


Washing Clothes in Cold Water

Cadmus estimated the energy savings from participants washing clothes in cold water in two steps:

1. Estimated the energy usage of a clothes washer (using algorithms from the PA TRM)160

2. Weighted the results based on WRAP PY10 survey results

Taking Shorter Showers

Cadmus assumed that participants who said they take shorter or fewer showers take a five-minute

shower every time. Cadmus estimated shower energy use using section 2.3.9 in the PA TRM, which

concerns low-flow showerheads but was a good proxy after adjusting the flow rate to be constant (the

weighted flow rate for WRAP participants), then added a term to subtract the energy education

recommendation for shower length from the default.161

Ex Post Savings Calculation Methodology for LED giveaway events

The ICSP organized a total of four community events in PY10 and distributed a total of 2,450 packs of

LED bulbs (total of 5,800 LED bulbs) in these community events:

• Community Event 1 at Hazleton Young Women's Christian Association (YWCA). For this

community event, the ICSP selected a nonprofit agency that provides community programs for

kids and families and mailed post cards to 1,537 OnTrack customers. 500 packs of LED bulbs (4

LED bulbs per pack; total of 2000 LED bulbs) were provided to participants. This event was open

to the public.

• Community Event 2 at Bethlehem Cops and Kids. The ICSP selected this community event for

community visibility at a well-attended annual event targeting families. The ICSP mailed post

cards to 5,889 OnTrack customers. 1,750 packs of LED bulbs (2 LED bulbs per pack; total of 3,500

LED bulbs) were provided to participants. This event was not open to the public.

• Community Event 3 at Lancaster Community Action. For this community event, the ICSP

selected an agency that provides a suite of services to low-income customers. The ICSP mailed

post cards to 2,026 OnTrack customers. 100 packs of LED bulbs (2 LED bulbs per pack; total of

200 LED bulbs) were provided to participants. This event was not open to the public.

• Community Event 4 at Lancaster Habitat for Humanity. For this community event, the ICSP

selected an agency that provides service to low-income customers in need. The ICSP provided

LED bulbs and WRAP brochures and had an information table during the event. It provided 100

packs of LED bulbs (1 LED bulb per pack; total of 100 LED bulbs) to participants. This event was

not open to the public.

160 Section 2.3.6 of the PA TRM concerns the water heater temperature setback. One component in the algorithm estimates savings from the clothes washer. Cadmus used these savings to estimate consumption of a clothes washer.

161 The PA TRM groups like terms and takes the difference of the variables that are changed. In this instance, Cadmus set the flow rate to be constant and changed the time of the showers.


Cadmus received all the details of the LED giveaway events from the ICSP including the following:

• Locations (zip codes) of the events

• How the locations were chosen for low-income community events

• Description of the collaborating agencies

• Number of invitation postcards sent to OnTrack participants by the ICSP per event

• Number of participants attending each event

• Marketing materials including postcards sent to customers and event posters

For each LED giveaway community event, the ICSP targeted and sent postcards to OnTrack participants,

who are all income eligible (at or below 150% of the federal poverty level).

Cadmus determined verified energy savings from LED bulbs using the PA TRM entries.162 Following this

ex post savings calculation methodology, Cadmus verified total energy savings from LED giveaway

community events as 193 MWh/yr.

Ex Post In-Service Rates Calculation Methodology

Cadmus fielded two surveys in PY10 to calculate ISRs for six products: LEDs, LED nightlights, kitchen

aerators, bathroom aerators, showerheads, and Tier 2 advanced power strips. Cadmus found no

meaningful difference between responses to the two surveys and aggregated responses in PY10.

Ex Post Demand Reduction

For all projects, Cadmus followed the PA TRM to estimate demand savings. To calculate the kW/yr

reduction at the generator, Cadmus applied the residential line loss factor of 0.0833.

J.1.2 Database Review Findings

Database quality control review assessed the completeness of fields necessary to conduct the

participant telephone surveys and to verify that items recorded in the PPL Electric Utilities’ tracking

database for each job sampled matched the items installed from the ICSP’s ERMS database. Cadmus

found no major issues with either database.

J.1.3 Participant Counts

Cadmus used the unique utility account number as the participant. During the review of extracts from

the PPL Electric Utilities’ tracking database, Cadmus found cases where the same utility account number

was associated with multiple job types. In some cases, this was because a baseload turned into a low-

cost job when contractors added products. Table J-5 presents the participation counts for WRAP in

PY10.

162 Section 2.1.1 of the PA TRM.


Table J-5. PY10 WRAP Participant Counts

WRAP Job Type No. of

Accounts Reported

Participants Difference Notes

Baseload 6,674 6,671 3 One Baseload job turned into a Low-Cost job and two Baseload jobs were also reported as Manufactured Home Initiative jobs

Low-Cost 4,670 4,666 4 Four Low-Cost jobs were also reported as Manufactured Home Initiative jobs

Full-Cost 1 1 0


1,621 1,621 0


53 53 0

LED Giveaway N/A 2,450 N/A 2,450 packs of LED bulbs (total of 5,800 LED bulbs) were provided to 2,450 households

Program Total 13,019 15,462 7

J.1.4 Records Review Findings

This section presents the key findings from Cadmus’ review of records in the form of Table J-6. These,

along with the ISRs of key products, are the reasons for the differences between reported and verified

savings.

Table J-6. PY10 Record Review Findings

Stratum Product Issue No. of Jobs Effect on Savings

Baseload

LEDs

Room reported as overall household, but audit records indicated specific room

2 Increase

Hours of use assumed 90% of bulbs are not efficient when audit records showed 90% were efficient

1 Decrease

Tier 2 power strips

Installed in entertainment center but less than three devices plugged in

3 Decrease

Three Tier 2 power strips installed in an entertainment center with one device plugged into each; study TRM references based savings estimate on two power strips

1 Decrease

Three Tier 2 power strips installed in an entertainment center with at least three devices plugged into each; study TRM references based savings estimate on two power strips. Cadmus is allowing savings for three "entertainment centers" but not four.

1 No effect


Stratum Product Issue No. of Jobs Effect on Savings

Low-Cost

LEDs

Room type reported as "other" for all bulbs. "Other" should only be used in conjunction with specific rooms listed in the TRM. In this case, "overall household" should be used

1 Increase


1 Decrease

Tier 2 Power Strips

Three Tier 2 power strips installed in an entertainment center with at least three devices plugged into each; study TRM references based savings estimate on two power strips. Cadmus is allowing savings for three "entertainment centers" but not four.

1 No effect

Showerheads, aerators, TSVs

Home type reported as single family, but audit records show it's a multifamily home

5 Decrease

Water heater pipe insulation

Fifteen ft of pipe insulation installed, but, per source in TRM, savings capped at 10 ft

1 Decrease

LED night light

Inspection record does not show night light installed 1 Decrease

Manufactured Home Initiative

LEDs

Audit records show LED quantities installed in rooms were switched

1 Increase

Room type reported as "other" for all bulbs. "Other" should only be used in conjunction with specific rooms listed in the TRM. In this case, "overall household" should be used

1 Increase


3 Decrease

Tier 2 power strips

Four Tier 2 power strips installed in "entertainment centers", which goes past the study the TRM references. Cadmus is allowing savings for three "entertainment centers" but not four.

1 Decrease

Showerheads, aerators, TSVs

Home type reported as single family, but audit records show it's a multifamily home

1 Decrease

Furnace whistle

Furnace whistle installed in home with gas furnace and no central AC

1 Decrease

Master-Metered Multifamily

LEDs Hours of use assumed 90% of bulbs are not efficient when audit records showed 90% were efficient

1 Decrease

J.1.5 Energy Education Savings Analysis Findings

Table J-7 shows the energy savings recommendations considered in estimating energy education

savings, the behavioral element that education could change, the PA TRM reference, the WRAP

participant survey results, and the per-unit energy and demand savings in each half of the year. In PY10,

the verified energy education savings estimate is 89.02 kWh/yr per household.

The ex ante assumption was 160 kWh/yr; verified savings were lower than the ex ante savings.


Table J-7. Verified Energy Education Savings and Assumptions Summary Table

Energy Savings Recommendation

Behavioral Assumption

PA TRM Reference

Ex Post Verified Savings

kWh/yr kW

Adjust Thermostats – Summer Participants lower their thermostat in the winter and raise it in the summer

Programmable Thermostats – Section 2.2.8

2.01 -

Adjust Thermostats – Winter 80.69 -

Wash Clothes in Cold Water Participants increase the number of loads of laundry they wash in cold water

Water Heater Temperature Setback – Section 2.3.6

5.85 0.0005

Take Shorter Showers Participants decrease the duration of each shower

Low Flow Showerheads – Section 2.3.9

0.47 0.00004

Total (1) 89.02 0.0005

(1) Each component is summed to get the total.

Table J-7 shows that the main driver in the energy education savings was adjusting thermostats in the

winter—where 74% of those surveyed said they adjusted their thermostat in the winter. This

component has consistently driven the energy education savings.

Table J-8 shows the results for Cadmus’ estimation of the overall percentage change of WRAP

participants’ who took shorter showers and wash clothes in cold water. Similar to previous years, many

people are already taking some of these actions and many people are taking no action, both of which

combine to have a marginal effect on overall savings.

Table J-8. Calculation and Results of Energy Education Percent Change Components

Energy Savings Recommendation Percent Before Percent After Percent Change

Of Those Who Took Action

Taking Shorter Showers 52.6% 54.5% 1.9%

Wash Clothes in Cold Water 61.9% 76.0% 14.1%

Overall Percentage Change

Energy Savings Recommendation and Action Percent Change Percent of Population (1) Overall Percent Change

Taking Shorter Showers - Action 1.9% 20.6% 0.4% (2)

Taking Shorter Showers - No Action 0.0% 79.4%

Wash Clothes in Cold Water - Action 14.0% 17.4% 2.4% (2)

Wash Clothes in Cold Water - No Action 0.0% 82.6%

(1) This is the percentage of the surveyed population who took action—i.e., 20.6% took shorter showers and the average increase in shorter showers among those was 1.9%. (2) Cadmus calculated this by taking the weighted average of the percent change and the proportion of the population that either took action or did not. By definition, those who took no action had a percent change of zero.


J.1.6 In-Service Rates

Cadmus surveys addressed six key products—LEDs, LED nightlights, kitchen aerators, bathroom aerators,

showerheads, and Tier 2 advanced power strips. Cadmus found no meaningful difference between

surveys conducted in Q1-Q2 participants and Q3-Q4 participants. Thus, Cadmus aggregated survey

results to compute the PY10 ISR for each of the products.

Table J-9 compares the ISRs from all of PY8, PY9 Q1-Q2, PY9 Q3-Q4, and PY10.

Table J-9. Comparison of ISRs in Phase III

Product PY8 (1) PY9 Q1-Q2 PY9 Q3-Q4 PY10

LEDs 94% 97% 99% 100%

LED Nightlights 69% 83% 96% 96%

Kitchen Aerators 78% 86% 95% 98%

Bathroom Aerators 61% 76% 90% 93%

Showerheads 88% 91% 93% 94%

Tier 2 Advanced Power Strips 44% 66% 88% 85%

(1) ISRs collected via site visit data.

J.2 Process Evaluation

J.2.1 Additional Findings

This section includes additional process evaluation findings.

Program Delivery

Cadmus reviewed the logic model and determined that WRAP is operating as expected. Table J-10 lists

the outcome of the logic model review.


Table J-10. Winter Relief Assistance Program Logic Model Review

Expected PY10 Outcome Topics Actual PY10 Outcome

Marketing and referrals from other low-income programs (Act 129

and Universal Services) identify participants, establish participants’

eligibility, conduct energy audits, conduct improvement-eligibility

assessments, install energy-efficient equipment, provide energy

education, and generate referrals to other organizations for

participant households.

Program Activities Delivered program activities

as expected.

The ICSP enrolls income-qualified participants, completes audits,

installs energy-saving products, and serves clients.

Outputs Produced

by Program

Activities

Delivered outputs as

expected.

Increase program awareness, install energy-efficient equipment in

participant homes, increase participant knowledge of energy

efficiency and conservation, and provide access to other needed

services.

Short-Term

Outcomes

Produced short-term

outcomes as expected.

Energy savings accrue from participant households through

installation of efficient equipment.

Intermediate

Outcomes

Program on track to meet

intermediate outcomes.

Energy savings continue to result from energy-efficient equipment

upgrades and conservation behaviors in the participating low-

income population.

Long-Term

Outcomes

Program on track to meet

long-term outcomes;

Cadmus will assess at the

end of Phase III.

Participant Profile

In the participant phone surveys, Cadmus collected demographics and home characteristics. The

participants’ homes had the following characteristics (n=150).163

• Single-family detached residence (25%)

• Attached house (townhouse, rowhouse, or twin) (20%)

• Mobile or manufactured home (37%)

• Multifamily apartment or condo building with four or more units (17%)

• Other (1%)

WRAP respondents have the following level of education (n=147):164

• Less than high school diploma or equivalent (16%)

• High school diploma or equivalent (54%)

• Technical or business school certificate/two-year college degree/some college (23%)

163 Three percent of the participants preferred not to answer this question.

164 Six percent of the participants preferred not to answer this question.


• Four-year college degree/bachelor’s degree (5%)

• Graduate or professional degree/masters or doctorate degree (1%)

J.2.2 Survey Approach

Sample Attrition

Table J-11 lists the total number of records submitted to the survey subcontractor and the outcome

(final disposition) of each record.

Table J-11. PY10 WRAP Sample Attrition Table for Participant Telephone Surveys


Population (number of unique jobs) [1] 11,423

Removed: inactive customer, completed survey in past three months, on "do not contact" list, selected for a different survey, duplicate contact

2,152

Incomplete or invalid phone number or email 936

Survey Sample Frame (sent to subcontractor for telephone survey calls) 8,335

Not attempted [2] 5,013


Non-working, wrong number, business, language barrier 465


2,688


14

Completed Surveys (telephone) 155

Telephone Response rate 5%

(1) Total records do not include master-metered multifamily building tenants as there is no contact data available for tenants. Therefore, no surveys were conducted with participating tenants in master-metered multifamily buildings. Cadmus evaluated satisfaction through interviews with master-metered multifamily building property managers. (2) Selected for sample but target was reached before attempted.

Appendix K. Evaluation Detail – Energy Efficient Home Program PPL Electric Utilities | K-1

Evaluation Detail – Energy Efficient Home Program

This appendix details the methodologies and results for the Energy Efficient Home Program evaluation

activities.

K.1 Gross Impact Evaluation

K.1.1 Ex Post Verified Savings Methodology

Cadmus assessed savings for the sampled units, calculated stratum-level realization rates, then applied

the realization rates to the population total ex ante savings within each stratum to estimate the stratum

total ex post savings. Next, Cadmus summed the stratum total ex post savings to derive the program

total ex post savings and calculated the program realization rate by dividing the program total ex post

savings by the program total ex ante savings.

Cadmus calculated realization rates, standard errors, and precision for the total ex post savings

estimates using formulas provided in the Uniform Methods Project’s sampling chapter and the Phase III

Evaluation Framework using sampling weights (wi) proportional to the sampling probability of each

unit.165 In stratified sampling, the weights are equal to the stratum population size (Nh) divided by the

stratum sample size (nh), that is, whi = Nh/nh, for stratum h and unit i. 166

Cadmus calculated the relative precision of the program’s total ex post savings and realization rate

estimates at a minimum of 85% confidence. It designed the sample with a 15% precision target for the

program’s total energy savings to achieve PPL Electric Utilities’ 85/15 program target for confidence and

precision, as stipulated in the Phase III Evaluation Framework. The following sections discuss detailed

methodology for each component of the Energy Efficient Home Program.

K.1.2 New Homes

New Homes Methodology

Cadmus calculated the ex post evaluated gross savings by summing the ex post weather-sensitive and

non-weather-sensitive savings. First, Cadmus calculated ex post energy savings and demand reductions

for the non-weather-sensitive equipment (heat pump water heaters, refrigerators, dishwashers, clothes

washers and lighting) according to the individual appliance and lighting algorithms in the PA TRM. For

these equipment types, Cadmus used data collected during site visits in PY8 and by the ICSP’s

subcontractor in PY9.

165 National Renewable Energy Laboratory. Chapter 11: Sample Design Cross-Cutting Protocols. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures. Prepared by Cadmus. April 2013. Available online: http://energy.gov/sites/prod/files/2013/11/f5/53827-11.pdf


http://energy.gov/sites/prod/files/2013/11/f5/53827-11.pdf


For weather-sensitive products, Cadmus examined the REM/Rate files and ex ante savings provided by

the ICSP’s subcontractor to determine if inputs to the simulations and savings were reasonable.

Cadmus selected a simple random sample from the 1,002 unique projects in the New Homes component

to evaluate 40 REM/Rate energy models. Of these 40 sites, eight were units in a multifamily building; the

ICSP’s subcontractor modeled them as single-family homes and each unit met the same efficiency

requirements. Cadmus also modeled these units as single-family homes (with adiabatic walls and other

inputs as needed). Cadmus then determined ex post energy savings and demand reductions for

envelope and HVAC equipment using output from the REM/Rate simulations that calculated heating and

cooling energy savings.

Cadmus used REM/Rate version 15.7 and incorporated the built-in baseline reference home that RESNET

specifically designed for PPL Electric Utilities’ New Home component. Cadmus also compared the input

data (pertaining to mechanical equipment, lighting, and building envelope) for the ICSP’s

subcontractor’s REM/Rate files against data collected during Cadmus’ site visits for 20 homes in PY8.

Four multifamily units in the evaluation sample also installed photovoltaic (PV) panels on the roofs.

Because the ICSP’s software calculated PV system savings, the ICSP reported these energy savings to PPL

Electric Utilities. The PA TRM does not address PV systems installed on new homes. More detail about

Cadmus’ methodology to evaluate savings from PV systems can be found in the PY9 Annual Report,

Appendix J.167

The next sections discuss Cadmus’ approach to verifying energy savings and demand reduction.

Method to Verify Lighting and Appliance Energy and Demand Savings

To calculate energy and demand savings for light bulbs and appliances, Cadmus used data from PY8 site

visits, the sampled PY10 REM/Rate files, and the PA TRM. Cadmus did not conduct site visits in PY10.

Lighting. In the PY8 lighting inventory data, Cadmus observed that verified energy savings correlated

closely to the documented percentage of high-efficiency lamps in REM/Rate files so developed a

regression of PY8 data to estimate energy and demand savings. This method corrects for the known

variables in REM/Rate: home size and percentage of high-efficacy lamps. The savings equations used in

PY10 are the same as those used in PY9, and the equations are contained in the PY9 Annual Report,

Appendix J.168




K.1.3 New Homes REM/Rate Modeling Findings

The ICSP reports energy savings and demand reduction for the New Homes component directly from the

REM/Rate software output that HERS raters use to verify that the home meets program requirements.

The module calculates energy and demand savings for all components of the homes; however, the

methods and equations used by REM/Rate differ from the PA TRM for most components. The ICSP

worked with software providers in PY10 to address the issues.

Cadmus found that ex post and ex ante heating and cooling savings agreed exactly.

For lighting and appliances, Cadmus found that the ex post savings were lower than ex ante estimates.

This is due to differences in the built in assumptions in REM/Rate as compared to the assumptions the

PA TRM uses. REM/Rate uses an outdated methodology for lighting that overestimates energy savings.

This is due to a difference in the baseline lighting assumptions between the PA TRM and REM/Rate.

REM/Rate also calculates savings for appliances even if the appliances were not installed when the

home was rated by HERS raters. Additionally, REM/Rate does not contain all necessary data to

accurately define the baseline appliances and verify appliance ENERGY STAR criteria. This is especially

true for refrigerators where the size and configuration of the installed refrigerator were not included in

the REM/Rate inputs, but the data were needed to verify energy savings.

For clothes washers, Cadmus found that there are additional savings REM/Rate is capturing based on

the labeled energy rating (LER). For 13 homes, the default LER for the design home was less than for the

reference home, resulting in additional savings. However, it was impossible to verify whether a clothes

washer was actually installed because the REM/Rate model includes a clothes washer and the HERS

raters who generate the model do not document clothes washer LERs.

For clothes dryers, Cadmus found an error in the savings calculations originating as an error in either the

reference home or REM/Rate’s MEF/IMEF (Modified Energy Factor/Integrated Modified Energy Factor).

Cadmus verified that this calculation created artificially high savings for clothes dryers.

There were additional ex ante savings for water heaters as well. These savings were calculated as the

difference between the reference home and design home energy consumption for water heaters.

Twenty-two of the 40 sampled sites claimed savings for water heaters. Based on the REM/Rate data,

none of these water heaters qualified as a heat pump water heater. Since the PA TRM only provides

savings for heat pump water heaters there were no ex post energy savings.

K.1.4 Efficient Equipment

Database and Records Review Findings

This section summarizes findings from Cadmus’ database and records review of Efficient Equipment

components that were not discussed in section 13.2.2: Gross Impact Evaluation Results.

• No errors were found in the review of pool pumps, air source heat pump, or central air

conditioner participants. The energy realization rates for these equipment offerings were 100%.


• The overall energy realization rate for smart thermostat projects was 99%, despite Cadmus

findings data entry errors in the PPL Electric Utilities tracking database.

• The overall realization rate for water heater fuel switching projects was 98% for energy and

demand. Cadmus found the following errors:

▪ The baseline energy factor in the rebate application did not match the tracking database

(seven total instances). Cadmus used either the reported energy factor of the electric water

heater (if it was still working at the time of replacement) or the PA TRM default for water

heater energy factor based on tank size. In three cases, the customer listed the model

number and energy factor of the electric water heater in the rebate application, and these

did not match the reported energy factor. In four instances, the reported baseline tank size

did not match the baseline tank size in the tracking data.

▪ In three instances, customers installed gas tankless water heaters with energy factors lower

than reported. This increased the fossil fuel consumption.

▪ In three instances, either the zip code or the installation date was missing from the rebate

application.

▪ Two fields in the tracking database were difficult to interpret:

− Cadmus found the PPL Electric Utilities tracking database field called

“HEATINGSYSTEMTYPE” was filled out as “storage tank” or “heat pump” for all records.

It is likely this field was created for central heating system fuel switching projects and

not updated for water heater fuel switching.

− Cadmus found the PPL Electric Utilities tracking database field called

“WHATNEWELECTRICEQUIPREPLCE” was filled out as “permanent electric heat” for all

records. This field name is confusing because the baseline equipment, not new

equipment, is electric.

• The realization rate for dehumidifiers was 63% for energy and demand. The majority of the

errors were among dehumidifier projects. The ICSP used current ENERGY STAR standards for

dehumidifiers, 169 but Cadmus followed the PA TRM which uses the 2012 ENERGY STAR

standard.170 Cadmus discussed the same issue in the PY9 Annual Report.171 In addition, a few

records’ model numbers did not meet either the 3.0 or 4.0 ENERGY STAR specifications for

dehumidifiers. Cadmus applied zero ex post savings to these records.

• The realization rates for ductless heat pumps were 96% for energy and 97% for demand. With

ductless heat pumps, Cadmus found several issues with the AHRI numbers and the indoor and

169 EPA. ENERGY STAR Program Requirements for Dehumidifiers Version 4.0. October 25, 2016. Accessed online: https://www.energystar.gov/products/appliances/dehumidifiers/key_efficiency_criteria.

170 See section 2.4.8 of the PA TRM. This section uses ENERGY STAR Version 3.0.


https://www.energystar.gov/products/appliances/dehumidifiers/key_efficiency_criteria



outdoor unit model numbers. There were instances where the AHRI number was entered in the

model number field in the tracking database (the AHRI number is not required on the rebate

forms; the model number is, however). Cadmus confirmed that the error initiated in the ICSP’s

tracking data. In addition, these AHRI numbers were only for outdoor units, and AHRI numbers

for indoor units were not recorded. In some instances, rebate applications and contractor

invoices listed different model numbers, and model numbers could not always be located in the

AHRI directory.172 Cadmus had difficulty looking up specifications for indoor units because

model numbers were not recorded by the ICSP. Three ductless heat pump projects that were

not included in Cadmus’ records review sample had an outdoor unit recorded in the database

but no corresponding indoor units. Reported savings were zero. Cadmus removed these projects

from the count of verified participants.

• There were several errors in central heating equipment fuel switching projects due to the PPL

Electric Utilities tracking database applying fuel furnace equivalent full load hours (EFLH) instead

of the corresponding fuel boiler EFLH. The energy realization rate was 101%.

• There were several ASHP tune-up records that had missing ASHP cooling and/or heating

capacities. The energy realization rate was 100%.

• There was only one error for refrigerator projects. It is likely this was due to a data entry error in

the PPL Electric Utilities tracking database as the reported savings were twice that of similar

refrigerators. The energy realization rate was 100%.

K.1.5 Audit and Kits Installation Verification Methodology and Findings

In PY10, Cadmus conducted participant surveys to calculate ISRs for online assessment kit products in

the Audit and Kits stratum. These ISRs were also applied to in-home audit kit products. Cadmus

confirmed that rebated products were installed by asking in the online survey: “Is the [kit product] you

received in your kit currently installed in your home?” or “Of the [number of product distributed] [kit

product] you received in your kit, how many are currently installed in your home?” Two hundred and

thirty-one respondents answered the survey’s questions pertaining to installation verification.

ISRs determined in PY10 were lower than the ISRs used by the ICSP in reported savings calculations.

Both are summarized in Table K-1.

172 Air-Conditioning, Heating, and Refrigeration Institute. AHRI Directory of Certified Product Performance. Available online: https://www.ahridirectory.org/Search/SearchHome?ReturnUrl=%2f

https://www.ahridirectory.org/Search/SearchHome?ReturnUrl=%2f


Table K-1. PY10 Reported vs. Evaluated ISRs for In-Home Audit and Online Assessment Kit Products

Kit Product PY10 Reported ISRs

(In-Home Audit) PY10 Reported ISRs (Online Assessment)

PY10 Evaluated ISRs

LED light bulb 97% 95% 82%

Low-flow showerhead 88% 72% 44%

Bathroom faucet aerator 97% 51% 41%

Kitchen faucet aerator 97% 75% 42%

Pipe insulation 100% 100% 62%

K.1.6 Efficient Equipment and Weatherization

Installation Rates

In PY10, Cadmus used ISRs calculated in PY9 for dehumidifiers and ISRs calculated in PY8 for the

remaining products in the Efficient Equipment stratum as well as the Weatherization stratum. No

program changes were made that affected these products.

K.2 Process Evaluation

K.2.1 Additional Findings

This section includes additional process evaluation findings for the Energy Efficient Home Program.

Program Component Satisfaction

Table K-2 shows each satisfaction metric broken out by program component.

Table K-2. Satisfaction Detail by Program Component

Very

satisfied Satisfied

Neither satisfied nor dissatisfied

Not too satisfied

Not at all satisfied

Don’t know

Overall

Equipment (n=208) 71% 22% 1% 3% 2% 0%

Weatherization (n=38) 51% 30% 10% 5% 2% 1%

Online Assessment (n=243) 66% 24% 5% 5% 0% 0%

In-Home Audit (n=6) 100% 0% 0% 0% 0% 0%

Total (n=495) (1) 61% 26% 6% 4% 2% 1%

Clarity of application requirements

Equipment (n=206) 65% 23% 6% 3% 1% 2%



In-Home Audit (n=6) 50% 33% 17% 0% 0% 0%

Total (n=446) (1) 57% 24% 10% 3% 2% 4%


Very

satisfied Satisfied

Neither satisfied nor dissatisfied

Not too satisfied

Not at all satisfied

Don’t know

Information you have learned online from PPL Electric Utilities about how to save energy

Equipment (n=194) 62% 24% 12% 0% 0% 3%



In-Home Audit (n=6) 33% 50% 175 0% 0% 0%

Total (n=451) (1) 56% 27% 12% 2% 2% 2%

PPL Electric Utilities’ rebates for qualifying energy-efficient equipment and services

Equipment (n=207) 74% 21% 1% 2% 2% 0%



In-Home Audit (n=6) 67% 33% 0% 0% 0% 0%

Total (n=416) (1) 59% 20% 8% 4% 4% 5%

(1) Total may not sum due to rounding.

Desired Program Improvements (Additional Detail)

There are several ways PPL Electric Utilities can convert customers to promoters to raise satisfaction

with the program, particularly among online assessment participants for whom satisfaction has been

lowest year-to-year. Table K-3 shows participants’ suggested improvements by program component.

Several online assessment participants said that the energy-efficiency kit173 was insufficient for a variety

of reasons, including that the kit was “outdated,” did not have the components they wanted, and

included inadequate supplementary information. For example, one participant mentioned that “[the

energy efficiency tips] are probably adequate for someone who is new, but I did not learn anything

myself.”

In general, the responses from online assessment component participants indicated a strong desire to

achieve energy savings in excess of the savings enabled by the items and information provided by the

current kit. Several respondents stated that the energy efficiency products would be more effective if a

PPL Electric Utilities representative were to visit homes to conduct energy assessment tests and install

the kit products for participants. Adding ways for customers to customize the kits, such as adding two to

three pre-made options they can choose from, may help maximize the savings benefits realized by the

same kit items and increase ISRs. The items that participants most often requested that PPL Electric

Utilities include in the kit were a larger variety of LED bulbs (five participants), additional advanced

power strips (two participants), and handheld showerheads (two respondents). By broadening kit

product selection and helping participants better understand how each product is used, PPL Electric

Utilities might help convert online assessment passives and detractors to promoters.

173 Participants receive kits by mail after completing the online assessment.


In PY9, Cadmus recommended that the ICSP encourage trade allies to help customers fill out equipment

rebate forms to alleviate confusion and frustration with the rebate application process. The ICSP

reported that they are communicating with contractors, through their trade ally newsletter and trade

ally breakfasts. The ICSP considered other options to improve customer service among trade allies, such

as offering an incentive to trade allies for submitting the rebate form on behalf of the customer.

Additionally, in PY9, Cadmus recommended that PPL Electric Utilities add information on the customer-

facing program website to help explain the rebate forms better. PPL Electric Utilities did not take action

on this recommendation during PY10. Based on the results in Table K-3, specifically with customers

asking for more clarity on the rebate process, Cadmus has the same recommendation in PY10.

Table K-3. Suggested Improvements

Suggested Improvement

Frequency (Number of Respondents who Provided Suggestion)

In-Home Audit

Online Assessment

Weatherization Equipment Total

More clarity on/simplify application process – 2 of 63 2 of 13 23 of 89 27 of 166

More or other items in kit/help understand items in kit

– 27 of 63 – – 27 of 166

Advertise the program more 1 of 1 4 of 63 1 of 13 19 of 89 25 of 166

Increase speed of rebate process/make automated or online

– 3 of 63 2 of 13 15 of 89 20 of 166

Increase the rebate-eligible equipment – 7 of 63 2 of 13 10 of 89 19 of 166

Better customer service/follow up – 3 of 63 2 of 13 13 of 89 18 of 166

Increase rebate amount – – 4 of 13 12 of 89 16 of 166

More help understanding potential savings/more information to save energy

– 14 of 63 – – 14 of 166

Increase income limits/more options for renters

– 5 of 63 – – 5 of 166

Lower electricity rates/switch to TOU – 1 of 63 – – 1 of 166

Send kit as promised – 1 of 63 – – 1 of 166

Participant Profile

The PY10 customer surveys collected demographic information about Energy Efficient Home Program participants. The majority of respondents had the following characteristics:

• Lived in a single-family detached residence (80%; 363 of 453)

• Had an average household size of 2.3 people

• Averaged 58 years of age

• Had completed some college education or more (74%; 354 of 477)

• Had an annual household income of $45,000 or greater (66%; 254 of 383)

K.2.2 Survey Approach

Table K-4 lists total numbers of records contacted via online survey and the outcome (final disposition)

of each record. Additional details on the survey methodology are in Appendix R Survey Methodology.


Table K-4. Energy Efficient Home Sample Attrition Table

Description of Call Outcomes

Number of Records

In-Home Audit

Online Assessment

Weatherization Equipment

Population (number of unique jobs)(1) 376 6,244 1,639 3,391

Online

Removed: inactive customer, completed survey in past three months, on “opt out” list, selected for a different survey, duplicate contact, on “do not contact” list

262 447 1,014 305

Incomplete or invalid email address 13 1 108 729

Survey Sample Frame (email invitations sent) 101 5,796 517 2,357

Email was returned (bounce back), did not respond, opted-out, or did not complete survey

95 5,531 477 2,127

PPL Electric Utilities or market research employee 0 12 2 16

Cannot confirm equipment/not aware of participation 0 19 0 4

Completed Surveys 6 234 38 210

Online Response Rate 6% 4% 7% 9%

(1) Number of rebates available in PPL Electric Utilities’ tracking database at the time of the final survey effort.

Appendix L. Evaluation Detail – Appliance Recycling Program PPL Electric Utilities | L-1

Evaluation Detail – Appliance Recycling Program

This appendix presents details of the impact and process evaluation methodologies and results for the

Appliance Recycling Program.

L.1 Gross Impact Evaluation

L.1.1 Part-Use Factor Findings

Part-use is an adjustment factor specific to appliance recycling that is used to convert the unit energy

consumption (UEC) into an average per-unit gross savings. The UEC itself is not equal to the gross

savings for the following reasons:

• The UEC model yields an estimate of annual consumption.

• Not all recycled refrigerators would have operated year-round had they not been

decommissioned through the program.

As instructed in the Pennsylvania Phase III Technical Reference Manual (PA TRM), to calculate EDC-

specific part-use factors, Cadmus followed the methodology for recycled appliances described in the

Uniform Methods Project.174 Cadmus calculated part-use factors using PY10 participant survey data.

The part-use methodology relies on information from surveyed customers regarding pre-program usage

patterns, that is, how many months of the year prior to recycling that the appliance was plugged in and

running.

The final estimate of part-use reflects how appliances were likely to operate had they not been recycled

(rather than how they previously operated). For example, it is possible that a primary refrigerator

operated year-round would have become a secondary appliance and operated part of the time.

The methodology accounts for these potential shifts in usage types. Specifically, part-use is calculated

using a weighted average of the following prospective part-use categories and factors:

• Appliances that would have run full-time (part-use = 1.0)

• Appliances that would not have run at all (part-use = 0.0)

• Appliances that would have operated a portion of the year (part-use is between 0.0 and 1.0)

Cadmus calculated a weighted average part-use factor, representing the three participant usage

categories as defined by the appliance’s operational status during the year before recycling. For

example, Cadmus gave participants who did not use their appliance at all during the prior year a

part-use factor of zero, as no immediate savings were generated by the appliance’s retirement.




Using primary data gathered through the PY10 participant surveys, Cadmus took the following steps to

determine part-use:

1. Determined whether recycled refrigerators were primary or secondary units (treating all stand-

alone freezers as secondary units).

2. Asked participants who indicated they had recycled a secondary refrigerator or freezer if the

appliance had operated year-round, operated for a portion of the preceding year, or was

unplugged and not operated. Cadmus assumed all primary units operated year-round.

3. Asked participants who indicated they operated their secondary refrigerator or freezer for only

a portion of the preceding year to estimate the total number of months that the appliance

remained plugged in. This allowed the calculation of the portion of the year in which the

appliance remained in use. Cadmus determined that the average refrigerator, operating part-

time, had a part-use factor of 0.33, or four months. Freezers operating part of the time had a

part-use factor of 0.29, or three-and-a-half months.

These three steps resulted in information about how refrigerators and freezers operated prior to

recycling, as shown in Table L-1.

Table L-1. Historical Part-Use by Appliance Type (1)

Usage Type and Part-Use Category Percent of Recycled Units Part-Use Factor

Secondary Refrigerators Only n = 126

Not in Use 10% 0

Used Part Time 20% 0.33

Used Full Time 70% 1

Weighted Average 0.77

All Refrigerators (Primary and Secondary) n = 270

Not in Use 5% 0




All Freezers n = 90

Not in Use 13% 0




(1) Calculated using primary customer survey data from PY10 (2018-2019)

In many cases, the way an appliance was used historically (prior to being recycled) is not indicative of

how the appliance would have been used had it not been recycled. To account for this, Cadmus next

asked surveyed participants how they would have (likely) operated their appliances had they not

recycled them through the program. For example, if surveyed participants said they would have kept a

primary refrigerator in the program’s absence, Cadmus asked if they would have continued to use the

appliance as their primary refrigerator or would have relocated it, using it as a secondary refrigerator.


Participants who said they would have discarded their appliance independent of the program were not

asked about the future usage of that appliance, because that would be determined by another

customer. Since the future use type of discarded refrigerators is unknown, Cadmus applied the weighted

part-use average of all units (0.89) for all refrigerators that would have been discarded independent of

the program. By using this approach, Cadmus acknowledges that the discarded appliances might be

used as either primary or secondary units in the would-be recipient’s home.

Cadmus then combined the part-use factors shown in Table L-1 with participants’ self-reported actions

had the program not been available. This resulted in the distribution of likely future usage scenarios and

corresponding part-use estimates. The weighted average of these future scenarios, shown in Table L-2,

produced the part-use factor for refrigerators and freezers.

Table L-2. Prospective Part-Use by Appliance Type (1)

Use Prior to Recycling Likely Use Independent of Recycling Part-Use Factor Percentage of Participants

Primary Refrigerators

Kept (as primary unit) 1 8%

Kept (as secondary unit) 0.76 4%

Discarded 0.89 40%

Secondary Refrigerators Kept 0.76 18%

Discarded 0.89 30%

Overall 0.87 100%

Freezers Kept 0.80 34%

Discarded 0.80 66%

Overall 0.80 100%


Applying the part-use factors from Table L-2 to the modeled annual consumption from Table L-3 yields

the average gross per-unit energy savings. Table L-3 shows that the average gross savings for

refrigerators is 961 kWh/yr and savings for freezers is 708 kWh/yr.

Table L-3. Part-Use Adjusted Gross Per-Unit Savings

Appliance Average Per-Unit Annual

Energy Consumption (kWh/Year)

Part-Use Factor (1) Adjusted Per-Unit Gross

Energy Savings (kWh/Yr)

Precision at 90% Confidence

Refrigerators 1,104 0.87 961 11%

Freezers 885 0.80 708 23%


L.1.2 Regression Variable Findings

Table L-4 summarizes program averages or proportions determined through primary data gathering for

each open variable in the TRM regression equation and compares these to the TRM default values. The

TRM default values were used to calculate the reported savings for appliances recycled in PY10.


Table L-4. UEC Input Comparison for Refrigerator and Freezer Savings Algorithms

Equipment Independent Variable TRM Default PY10 EDC Data

Gathering Value

Refrigerator Recycling

Appliance Age (years) 29.41 22.8

Dummy: Manufactured Pre-1990 35% 28%

Appliance Size (cubic feet) 18.34 18.78

Dummy: Single-Door Configuration 5% 5.4%

Dummy: Side-by-Side Configuration 19% 21%

Dummy: Percent of Primary Usage (in absence of program) 65% 54%

Interaction: Located in Unconditioned space x CDDs 0.36 0.63

Interaction: Located in Unconditioned space x HDDs 2.08 4.38

Freezer Recycling

Appliance Age (years) 37.49 27.7

Dummy: Manufactured Pre-1990 60% 49%

Appliance Size (cubic feet) 15.74 15.71

% of appliances that are chest freezers 28% 33%

Interaction: Located in Unconditioned space x HDDs 4.93 6.86

Interaction: Located in Unconditioned space x CDDs 0.843 1.000

L.2 Net Impact Evaluation

L.2.1 Net-to-Gross Methodology

Cadmus used the following formula to estimate net savings for recycled refrigerators and freezers:

𝑁𝑒𝑡 𝑠𝑎𝑣𝑖𝑛𝑔𝑠 = 𝐺𝑟𝑜𝑠𝑠 𝑆𝑎𝑣𝑖𝑛𝑔𝑠 − 𝐹𝑟𝑒𝑒𝑟𝑖𝑑𝑒𝑟𝑠ℎ𝑖𝑝 𝑎𝑛𝑑 𝑆𝑒𝑐𝑜𝑛𝑑𝑎𝑟𝑦 𝑀𝑎𝑟𝑘𝑒𝑡 𝐼𝑚𝑝𝑎𝑐𝑡𝑠 −

𝐼𝑛𝑑𝑢𝑐𝑒𝑑 𝑅𝑒𝑝𝑙𝑎𝑐𝑒𝑚𝑒𝑛𝑡 + Spillover

Where:

Evaluated Gross Savings = The evaluated in situ unit energy consumption (UEC) for the

recycled unit, adjusted for part-use

Free ridership and

Secondary Market Impacts = Program savings that would have occurred in the program’s

absence

Spillover = Non-programmatic savings induced by the program

Secondary market impacts require a decision-tree approach to calculating and presenting net savings.

The decision tree—populated by the responses of surveyed participants—presents savings under all

possible scenarios concerning the participants’ actions regarding the recycled equipment. Through these

scenarios, Cadmus used a weighted average of savings to calculate the net savings attributable to the

program. This section presents specific portions of the decision tree to highlight each aspect of the net

savings analysis. The complete decision trees are presented at the end of this section.


Self-Report Survey

Cadmus used self-report surveys to assess net savings for the Appliance Recycling Program, following

the Evaluation Framework’s recommended common method for assessing free ridership.175 The SWE

team reviewed and approved the survey prior to fielding.

L.2.2 Net-to-Gross Ratio Sampling

Participant and Nonparticipant Surveys

Cadmus completed an online survey of 368 participants, as shown in Table L-5. The survey asked

questions to assess program satisfaction and gather data to estimate net savings.

Table L-5. PY10 Participant Survey Sampling Strategy

Survey Mode and Audience Population

Size(1) Final Sample Population

Target Sample Size


Response Rate

Online Participant Survey - Freezers

1,056 313 All available 93 30%

Online Participant Survey - Refrigerators

4,056 1,211 All Available 275 23%

Total Surveys Completed N/A N/A N/A 368 N/A (1) Population and final sample reflect number of participants selected for the sample. For participants who recycled multiple appliances, Cadmus randomly selected one appliance per participant.

Because freezers account for fewer units in the participant population, anyone who recycled both a

freezer and a refrigerator was included in the freezer sample frame. Cadmus cross-checked the

refrigerator and freezer sample frames to make sure that no one was included in both. Because room air

conditioners are picked up as an additional service, Cadmus did not generate a separate sample for

them.

See Appendix R Survey Methodology and Section L.3.2 Survey Approach for more details about survey

methodology, contact instructions, and sample attrition.

L.2.3 Net-to-Gross Findings

Free Ridership

Cadmus used self-report surveys to assess net savings for the Appliance Recycling Program, following

the Evaluation Framework’s recommendation to use the UMP guidelines for estimating program net

savings. Therefore, Cadmus followed the UMP guidelines updated in 2017, which recommend excluding

induced replacement as a net savings adjustment.176





Cadmus categorized participants’ self-reported discard methods as kept, transferred, or discarded, with

discarded indicating free ridership (i.e., their action would have led to removal of the appliance from the

grid without program intervention).

Table L-6 shows the percentage of participant units that would have been kept or discarded.

Table L-6. Final Distribution of Kept and Discarded Participant Appliances

Stated Action Absent Program Indicative of Free Ridership Refrigerators

(n=249) Freezer (n=80)

Kept No 34% 40%

Discarded Varies by Discard Method 66% 60%

Total 100% 100%

Note: Refrigerator and freezer response counts do not include “Don’t know” or “Refused”

To mitigate possible response bias, Cadmus also included questions in the general population survey to

identify nonparticipants, defined as customers who disposed of a working, qualifying appliance outside

of the program during PY10. Only 16 respondents fit this definition. In PY8, Cadmus had identified 10

nonparticipants but did not include them in the PY8 analysis due to the small sample size. Cadmus

combined the PY8 and PY10 responses, for a total of 21 nonparticipants who reported how they had

disposed of an appliance outside of the program.177 The UMP guidelines state that smaller sample sizes

are acceptable for a survey of nonparticipants given the greater evaluation resources required to reach

nonparticipants.

The relatively low number of identified nonparticipants is not unexpected. Assuming that an equal

number of refrigerators stop being used each year (either due to customer decision or failure of the

appliance) and an expected useful life (EUL) of 20 years means approximately one in 20 households (5%)

discard a refrigerator annually. However, identifying nonparticipants is further complicated since

customers were asked additional questions only if they had discarded an operable (therefore program-

eligible) appliance. Because inoperable units are discarded very differently than operable units, this

critical detail reduces the likelihood of identifying a nonparticipant.

Nonparticipant responses were sorted into the same scenarios as participant responses based on their

self-reported actual discard action, excluding the option “kept the appliance.”

After Cadmus determined the discard actions of the participants (those who would not have kept their

appliance) and nonparticipants, Cadmus then determined the overall discard scenarios by averaging the

ratios for participating and nonparticipating respondents, as shown in Table L-7.

177 Four PY8 respondents and one PY10 respondent did not answer the question about how they discarded their working appliance.


Table L-7. Final Distribution of Kept and Discarded Participant Appliances

Appliance Discard/Transfer

Scenario Participant Survey

Percent Nonparticipant Survey Percent

Average

Refrigerator Disposed 60% 54% 57%

Transfer 40% 46% 43%

Freezer Disposed 67% 25% 46%

Transfer 33% 75% 54%

Secondary Market Impacts

If Cadmus determined that a participant would have transferred the unit that was recycled to another

customer on the grid in the absence of the program, it is important to account for what the would-be

acquirer might have done after the participant recycled the unit.

After estimating the share of transfer units subject to secondary market impacts, Cadmus used a

decision tree to calculate the average per-unit program savings net of their combined effect. Cadmus

integrated these values into a combined estimate of savings net of free ridership and secondary market

impacts for refrigerators. The process for freezers is identical.

Figure L-1 details Cadmus’ methodology for assessing the program’s impact on the secondary

refrigerator market and for applying the recommended midpoint assumptions when primary data were

unavailable. As evident, accounting for market effects results in three savings scenarios:

• Full per-unit gross savings

• No savings

• Partial savings (i.e., the difference between energy consumption of the program unit and the

new, standard-efficiency appliance acquired alternatively)

Figure L-1. Secondary Market Impacts—Refrigerators

To calculate the replacement UEC for refrigerators and freezers, energy consumption of a new,

standard-efficiency appliance was calculated using the ENERGY STAR Website taking the average energy


consumption of new comparably sized, standard-efficiency appliances with similar configurations as the

program units.178

Induced Replacement

The 2017 update to the UMP protocol dropped induced replacement due to the difficulty of measuring

the effect and the small impact on savings.179 Therefore, Cadmus did not adjust net savings for induced

replacement.

Spillover

Table L-8 shows the spillover equipment, quantities, and source of savings results for the stratums of the

Appliance Recycling Program. The following is a list of the energy efficient equipment types contributing

spillover savings, attributed to PPL Electric Utilities, and the source of the estimated energy savings used

in the spillover analysis.

Table L-8. Spillover Equipment, Reported Quantities, and Savings Sources

Equipment Quantity Per-Unit Savings

kWh/yr Savings Source

Air conditioning equipment 8 235 PY10 PPL Electric Utilities Gross Verified Savings

Attic or ceiling insulation (sq. ft.) 2,172 1 PY10 PPL Electric Utilities Gross Verified Savings

Clothes washer 11 44 PPL Electric Utilities TRM Feb 2017

Dishwasher 8 41 PPL Electric Utilities TRM Feb 2017

Freezer 15 22 PPL Electric Utilities TRM Feb 2017

Refrigerator 37 131 PY10 PPL Electric Utilities Gross Verified Savings

Walls insulation (sq. ft.) 5,188 2 PY10 PPL Electric Utilities Gross Verified Savings

Windows 43 13 PPL Electric Utilities TRM Feb 2017

178 ENERGY STAR. “Find and Compare ENERGY STAR Certified Refrigerators.” https://www.energystar.gov/productfinder/product/certified-residential-refrigerators/results


https://www.energystar.gov/productfinder/product/certified-residential-refrigerators/results



Table L-9 shows each variable in the spillover calculation by equipment type.

Table L-9. Appliance Recycling Program Equipment Spillover by Equipment Category

Variable Variable Description Refrigerator Freezer Source

A Online Survey Sample Size (n) 271[1] 93[1] Survey Data

B Total Online Survey Sample

Spillover kWh/yr Savings 10,721 10,822

Survey Data / Engineering Estimates

C Average SO kWh/yr Savings Per

Survey Respondent 39.6 116.4 Variable B ÷ Variable A

D Program Participant Population 8,124 2,047 Program Tracking Data

E SO kWh/yr Savings Extrapolated

to the Participant Population 321,402 238,192 Variable C × Variable D

F Evaluated Program Population

kWh/yr Savings 8,702,492 1,547,024

Evaluated Gross Impact Analysis

G Spillover Percent Estimate 4% 15% Variable E ÷ Variable F [1] Online survey respondents in Q3 were not asked all of the spillover questions so only respondents from Q1 and Q2 are included in the sample for spillover.

Final Net-to-Gross Ratio

As shown in Table L-10, Cadmus determined final net savings as evaluated gross per-unit savings less

per-unit free ridership, secondary market impacts, induced replacement kWh/yr, plus spillover.

Table L-10. PY10 NTG Ratios

Appliance Gross Verified

Per-Unit Savings

Free ridership and Secondary

Market Impacts

(kWh/yr)

Induced Additional

Savings (Spillover)

Net Per-Unit kWh/yr

NTG

Refrigerator 961 408 38 591 61%

Freezer 708 352 106 462 65%

The decision tree in Figure L-2 illustrates how Cadmus combined the percentage of units that would

have been kept from Table L-6 (non-free riders), the secondary market impact from Figure L-1, and

those who would have disposed of their appliance (free riders) used to calculate the final NTG ratio for

refrigerators. Figure L-3 shows the calculations for freezers.


Figure L-2. Refrigerator NTG Combined Decision Tree


Figure L-3. Freezer NTG Combined Decision Tree


L.3 Process Evaluation

L.3.1 Additional Findings

This section includes additional process evaluation findings for the Appliance Recycling Program.

Participant Profile

The customer surveys conducted in Q1 and Q3 of PY10 (2018-2019) collected demographic information

about Appliance Recycling Program participants. The majority of survey respondents had the following

demographic characteristics:

• Lived in a single-family detached residence (85%; 297 of 351)

• Had an average household size of 2.4 people

• Average 61 years of age

• Had completed some college education or more (74%; 256 of 344)

• Had an annual household income of $50,000 or greater (64%; 169 of 264)

L.3.2 Survey Approach

Survey Frequency

Cadmus conducted online surveys with Q1 and Q3 participants who recycled refrigerators and freezers

and who had viable email addresses.

Sample Attrition

Table L-11 lists the numbers of records submitted to the survey subcontractor and the outcome (final

disposition) of each record.

Table L-11. Appliance Recycling Program Online Survey Attrition


Records

Online Survey

Population (number of unique jobs) (1) 6,205

Removed (incomplete, inactive customer, completed survey in past 3 months, on "opt out" list, selected for a different survey, duplicate contact, on “do not contact” list)

745

Email was incomplete or invalid 2,382

Survey Sample Frame (email invitations sent to all eligible) 3,078

Email was returned (bounce back, did not respond, opted-out, or did not complete survey) 2,695

Not eligible (PPL Electric Utilities or market research employee, not aware of participation) 15


Online Response rate 12% (1) Number of rebates for refrigerators and freezers available in PPL Electric Utilities’ tracking database in the first and the third quarters of the year, at the time of the final survey effort.

Appendix M. Evaluation Detail – Energy Efficiency Kits and Education Program PPL Electric Utilities | M-1

Evaluation Detail – Energy Efficiency Kits and Education

Program

M.1 Gross Impact Evaluation

M.1.1 Methodology

Survey Methodology

Each kit distributed through the Energy Efficiency Kits and Education Program included a paper survey

for the participant to complete and mail back to the ICSP. These surveys collected the necessary data for

Cadmus to calculate ISRs and determine the participant actions taken because of the program. Survey

data were also used to calculate ex post per-unit savings for each item in the energy-savings kit.


Cadmus estimated ex post verified savings for the Energy Efficiency Kits and Education Program for each

stratum—agency or direct mail delivery channels—and for the program overall using the ICSP-reported

savings, paper survey responses, and data from enrollment cards collected by the ICSP.

Assigning Survey Ex Ante and Survey Verified Savings

Cadmus assigned survey ex ante and survey verified savings to program participants based on the

criteria listed in Table M-1.

Table M-1. Criteria for Assigning Survey Ex Ante and Survey Verified Savings

Criteria Survey Ex Ante

Savings (1)

Survey Verified

Savings

Whether the respondent answered the product-specific question(s) ✓ ✓

How the participant answered questions on the enrollment card about home

characteristics ✓ ✓

How the respondent answered the questions asking if products were installed ✓

How the respondent answered questions about actions taken that could result

in behaviorally based energy savings ✓

(1) Cadmus used the ICSP-reported ex ante savings for survey-ex ante savings based on the listed criteria. The ICSP

incorporated information from the enrollment cards when calculating reported ex ante savings.

Cadmus assigned survey-verified savings using information from the enrollment card, specifically water

heater configuration, clothes washing location, type of space heating, type of space cooling, and type of

home. The ICSP assigned reported savings based on the data uploaded to PPL Electric Utilities’ tracking

database. Although there should be no discrepancies between data in PPL Electric Utilities’ tracking

database and in the enrollment cards, Cadmus investigated both sources and confirmed the correct

information with the ICSP when the two sources did not match. Cadmus found several differences

between the database and the enrollment cards and verified with the ICSP that the enrollment cards

reflected the most accurate information. Cadmus therefore used information from the enrollment card

and not the database to assign survey verified savings.


Cadmus calculated realization rates for each stratum as the ratio of survey verified savings to survey ex

ante savings. Because the kit contains one survey that asks questions about each item, survey responses

for products may be correlated within customers. Cadmus accounted for these correlations by rolling

savings up to the kit level prior to calculating realization rates and precision.

Survey Verified Savings

Cadmus independently calculated survey verified savings per the PA TRM and the associated algorithms.

These algorithms involve open variables for which the ICSP or Cadmus can use either the default or the

option of “EDC data gathering.” Table M-2 lists the algorithm inputs and sources of the data collected.

Table M-2. 2016 PA TRM Open Variables

Product Survey Data Enrollment Card Kit Specification Sheet

LED ISR Bulb wattage

LED Nightlight ISR, Baseline Condition Bulb wattage

Low Flow Showerhead ISR Water heater fuel type,

type of home Low flow GPM

Kitchen Faucet Aerator ISR Water heater fuel type,

type of home Low flow GPM

Tier 2 Advanced Power Strip ISR,

Equipment plugged into power strip

Furnace Whistle ISR Home heating fuel type,

home cooling configuration

Adjusting Thermostat for Cooling in the Summer

ISR Home cooling configuration

Adjusting Thermostat for Heating in the Winter

ISR Home heating fuel type

Water Heater Temperature Setback

ISR Water heater fuel type,

laundry location

Ex Post Verified Savings

To calculate stratum-level ex post savings, Cadmus applied the stratum-level realization rates to stratum

ex ante savings and took the sum of stratum-level ex post savings to estimate the program-level ex post

savings. Cadmus calculated confidence and precision for the ex post savings and realization rate

estimates in each stratum and for the program as a whole.

M.1.2 Database Review Findings

Cadmus reviewed PPL Electric Utilities’ program tracking database for all PY10 records for Energy

Efficiency Kits and Education Program participants. It reviewed the PPL Electric Utilities account

numbers, kit numbers, and home characteristics and compared these to information from the

enrollment cards recorded in the ICSP’s electronic database to ensure that records were traceable

between both databases.


Prior to Cadmus’ review of the database, PPL Electric Utilities’ tracking database listed a total of 13,756

kits. As a result of the review, Cadmus decreased the total unique distributed (and not returned) kits to

13,564 program kits, representing 99% database accuracy for the program, as shown in Table M-3.

Table M-3. Accuracy of PY10 Data for Energy Efficiency Kits and Education Program

Sector Product PY10 Kits in PPL Electric Utilities’

Tracking Database (1) Database Accuracy PY10 Verified Kits

Low-Income Energy-savings kit 13,756 99% 13,564

(1) The number of unique kits in PPL Electric Utilities’ tracking database that were not indicated as returned.

As mentioned, the number of unique CSP job numbers in PPL Electric Utilities’ tracking database does

not necessarily reflect the unique number of distributed kits, nor does it identify all kits that were

returned in PY10. Cadmus verified 13,564 kits as distributed and not returned from the 13,932 unique

CSP job numbers provided in PPL Electric Utilities’ tracking database using these steps:

• 16 unique CSP job numbers were associated with kits returned in PY10 but initially distributed in

PY9. Cadmus assigned these kits negative ex post to counter the positive ex post savings they

were assigned in PY9.

• 176 unique CSP job numbers were associated with kits distributed and returned in PY10.

Cadmus assigned these kits 0 kWh/yr and 0 kW/yr ex post savings.

M.1.3 Survey Findings

Cadmus estimated ISRs for all products in the energy-savings kits. Table M-4 provides these ISRs and the

ISRs the ICSP used for planning. As in PY9, the difference in ISRs for LED bulbs is primarily driven by the

delivery channel of the reported ISR; the data were gathered through surveys that were included in the

kit in Phase II when the kits only included two bulbs. Cadmus observed that ISRs remain relatively high

until after the fourth bulb, when installations drop off dramatically, ranging from 68% to 56% for the

fifth and sixth bulbs. Results are similar across strata.

Also similar to PY9, Cadmus observed higher furnace whistle ISRs than used for planning by the ICSP.

Note that the evaluated furnace whistle ISR provided in the table does not include a fuel saturation rate,

consistent with the value provided as the ICSP planning ISR.


Table M-4. PY10 Energy Efficiency Kits and Education Program Product-Level ISRs

Product

Agency Direct Mail

Survey-Gathered ISR

ICSP Planning ISR

Survey-Gathered ISR

ICSP Planning ISR

LED Bulbs 78% 96% 83% 98%

First Bulb 93% 96% 98% 98%

Second Bulb 93% 96% 97% 98%

Third Bulb 86% 96% 92% 98%

Fourth Bulb 76% 96% 82% 98%

Fifth Bulb 66% 96% 68% 98%

Sixth Bulb 56% 96% 61% 98%

LED Nightlight 44% 87% 52% 92%

Low-Flow Showerhead 44% 64% 53% 72%

Kitchen Faucet Aerator 60% 63% 68% 75%

Furnace Whistle 30% 17% 35% 20%

Table M-5 provides ISRs for showerheads. To evaluate showerhead savings, Cadmus incorporated both

the ISR questions from the kit survey as well as information on the enrollment card regarding the

number of showers in participating homes. If respondents indicated installing two showerheads from

the survey but only listed one shower in their household on the enrollment card, Cadmus applied

savings for one showerhead. Few respondents indicated installing both showerheads, but more

respondents than in previous years installed the first showerhead (in PY9 the ISRs for showerheads were

64% and 60% for agency and direct mail strata, respectively). The combined ISRs for both showerheads

were still lower than reported, leading to a 63% and 75% realization rate for agency and direct mail kits,

respectively. Since showerhead savings represented 33% of all reported savings, differences in reported

and evaluated ISRs had large impacts on overall realization rates.

Table M-5. PY10 Energy Efficiency Kits and Education Program Showerhead ISRs

Showerhead

Agency Direct Mail

Reported ISR Evaluated ISR Reported ISR Evaluated ISR

Showerhead 1 64% 82% 72% 90%

Showerhead 2 64% 7% 72% 16%

As described in Section 15.2.2 Gross Impact Evaluation Results, the increase in realization rate between

PY9 and PY10 was driven by the saturation of kit types distributed to customers because the ICSP’s

planned per-unit energy education savings aligned closer to kits with water products than to kits

without water products. Table M-6 provides the evaluated per-unit energy education savings

participants achieved by delivery channel and kit type based on participant survey responses to key

questions. The ICSP reported per-unit energy education savings of 253 kWh/yr for all participants,

regardless of delivery channel or kit type.


Table M-6. Energy Education Savings by Delivery Method and Kit Type

Delivery Method Kit Type Average Evaluated

kWh/yr Average Evaluated

kW/yr Sample Size

Agency Electric 286.46 0.0222 121

Non-Electric 13.93 0.0062 74

Direct Mail Electric 342.76 0.0226 361

Non-Electric 51.27 0.0126 392

M.1.4 Behavior Savings Methodology

Cadmus estimated the impacts of electric consumption associated with behavior changes by participants

in the Energy Efficiency Kits and Education Program using calculations derived from a combination of

engineering estimates, secondary research, and survey data. These savings estimates were associated

with the following behavior changes:

• Lowering the water heater temperature

• Washing more loads of laundry in cold water

• Adjusting the home thermostat per the heating or cooling season

The next sections provide details about the algorithms Cadmus used to estimate savings for these three

behavior changes. Cadmus used the same energy education savings algorithms for participants of the

Low-Income Winter Relief Assistance Program (WRAP) in PY9. See Chapter 12 Winter Relief Assistance

Program for details.

Water Heater Temperature Reduction

The Energy Efficiency Kits and Education program encourages participants to reduce the temperature

setting of their electric water heater to save energy. Cadmus estimated savings for this action by

following the PA TRM engineering calculation.180 The first term in the equation corresponds to the

savings from tank losses, and the second term corresponds to savings from the clothes washer, as a

result of changing the water heater setting.

Cadmus applied energy and demand savings to survey respondents who indicated on the enrollment

card that the home had an electric water heater. Respondents who indicated the home did not have an

electric water heater received zero electric savings for water heater temperature setback.

Furthermore, Cadmus applied the clothes washer portion of savings (corresponding to the second term

in the equation) only to participants who indicated on the enrollment card that they had a washing

machine in their home or apartment. Respondents who wash their laundry at an on- or off-site public

laundry facility were not eligible to receive the clothes washer portion of water heater temperature

reduction savings.

180 Pennsylvania Public Utility Commission. Pennsylvania Technical Reference Manual. June 2016.


Table M-7 provides the per-respondent savings applied to eligible participants.

Table M-7. Electric Water Heater Temperature Reduction Savings

Unit Tank Loss Clothes Washer Total

kWh/yr 86.77 79.09 165.86

kW/yr 0.0070 0.0064 0.0134

Washing More Loads of Laundry in Cold Water

Cadmus estimated the savings associated with washing more loads of laundry in cold water, a behavior

encouraged by the Energy Efficiency Kits and Education Program. Cadmus estimated these savings by

following the equation provided in the Phase III PA TRM,181 in which the change in percentage of loads

washed in cold water before and after the program is applied to the energy savings achieved when

lowering the temperature of the water used by the clothes washer.

In PY10, the paper survey did not include a question appropriate for determining any behavior change

related to program participation. To determine the change in the percentage of loads washed in cold

water, Cadmus applied the average change estimated in the PY7 evaluation of the Energy Efficiency Kits

and Education Program (formerly the E-Power Wise Program).182 Because these respondents did not

report a change (by responding with the same pre- and post-percentage of loads washed in cold water),

Cadmus applied clothes washer savings to all survey respondents with an electric water heater and in-

home laundry, adjusting the starting temperature of water and input to the TRM savings equation

depending on whether the respondent had lowered the water heater setting. As such, Cadmus did not

double-count savings from water heater temperature setback.

Table M-8 provides the per-respondent savings applied to eligible participants.

Table M-8. Washing More Loads of Laundry in Cold Water Savings

Unit Lowered Water Heater Setpoint Did Not Lower Water Heater Setpoint

kWh/yr 81.95 96.04

kW/yr 0.0066 0.0077

Adjusting Thermostat for Heating and Cooling Season

The Energy Efficiency Kits and Education Program recommends to participants that they save energy by raising their thermostat setpoint for cooling in the summer and lowering their thermostat setpoint for heating in the winter. Cadmus applied the Phase III TRM savings equation183 for installing a programmable thermostat but used the energy-savings factors for heating and cooling from the Iowa





Energy Wise Program evaluations, 184 which better reflect expected savings from adjusting thermostats for the heating and cooling seasons.

M.2 Process Evaluation

M.2.1 Additional Findings

This section includes additional process evaluation findings.

Participant Profile

From the ICSP, Cadmus collected enrollment card data for all customers who received an energy

efficiency kit. Data included details about participant demographics and home characteristics. As shown

in Table M-9, the homes of a majority of participants had the following characteristics:

• Single-family homes or apartments with two or fewer units (54%) or row house or townhomes

(20%)

• Two or fewer occupants in home (53%)

• One shower in the home (74%)

• Laundry facilities inside the home (81%)

• One to three bulbs on for two or more hours a day (54%) and four to six bulbs on for two or

more hours a day (34%)

Cadmus found significant differences in the distribution of responses for each of the home

characteristics between participants who were served through the agency delivery channel and through

direct mail.185 Cadmus also investigated which responses were significantly different within each home

characteristic. These statistically significant differences in the proportions of agency and direct mail

demographics are shown in Table M-9 with asterisks (*). Differences are likely driven by delivery

channel. Customers who enroll in the program via a direct mailer have been specifically identified by PPL

Electric Utilities. Customers who walk into the agencies do so of their own accord.

184 Cadmus. Iowa 2014 Energy Wise Program. January 31, 2015. Prepared for Iowa Utility Association. Cadmus. Iowa 2015 Energy Wise Program. January 30, 2016. Prepared for Iowa Utility Association. Cadmus. Iowa 2016 Energy Wise Program. February 22, 2017. Prepared for Iowa Utility Association. Cadmus. Iowa 2017 Energy Wise Program. March, 2018. Prepared for Iowa Utility Association.

185 Cadmus performed a chi-square test of independence for each home characteristic and found p-values of less than 0.0001 for each home characteristic at the 90% confidence level.


Table M-9. Energy Efficiency Kits and Education Program Customer Survey Demographics by Delivery Channel

Home Type (1)

Participant Answers Direct Mail (n=8,855) Agency (n=4,044) Total (n=12,899)

Single/Twin/Apartment (2 units)* 59% 44% 54%

Apartment Building (3 or more units)* 11% 26% 16%

Row House/Townhouse 19% 22% 20%

Mobile Home/Trailer 11% 9% 10%

Number of Occupants in Home (1)


1* 21% 30% 24%

2* 32% 23% 29%

3 19% 19% 19%

4 15% 14% 15%

5 or more 13% 14% 13%

Number of Showers (1)


1* 67% 90% 74%

2* 28% 10% 23%

3 or More 5% 1% 4%

Laundry Facility Location (1)


Inside Home* 86% 69% 81%

Off-Site Laundry Facility* 8% 21% 12%

On-Site Common Laundry* 6% 9% 7%

Number of Bulbs on for Two or More Hours (1)


1-3* 48% 69% 54%

4-6* 37% 27% 34%

7-9* 10% 3% 8%

10 or more 6% 2% 4%

(1) Percentages may not total 100% due to rounding. (2) Asterisks (*) indicate that the agency and direct mail participants responded at statistically different rates, based on all pairwise two-sample t-tests for proportions at the 10% significant level with Bonferroni correction.

Appendix N. Evaluation Detail – Student Energy Efficient Education Program PPL Electric Utilities | N-1

Evaluation Detail – Student Energy Efficient Education

Program

N.1 Gross Impact Evaluation

For the Innovation Pilot cohort, Cadmus verified ex post savings and completed a database review.

N.1.1 Ex Post Verified Savings Methodology

Figure N-1 presents a flow diagram for the ex post verified savings methodology for the Innovation Pilot

cohort. The rest of this section describes the methodology in greater detail.

Figure N-1. Ex Post Verified Savings Methodology Flow Diagram

Reported Ex Ante Savings (Flow Diagram Items 1 and 2)

Cadmus collected reported savings for each product and kit distributed to the population of Innovation

Pilot cohort participants from PPL Electric Utilities’ tracking database. A part of Cadmus’ quality control

process for evaluating the SEEE Program was to understand how the ICSP calculated reported savings.

To do this, Cadmus verified that the ICSP calculated per-unit savings according to the planning ISRs and

Pennsylvania TRM inputs specified in the planning documentation Cadmus received during PY10. This

process ensured that Cadmus and the ICSP were not making drastically different assumptions in

assigning savings to program participants.


Survey Ex Ante and Survey-Verified Savings (Flow Diagram Items 3, 4, and 5)

Cadmus estimated stratum-level realization rates using individual survey responses for the sample of

program participants who returned a HEW. Cadmus assigned per-unit survey ex ante and survey-verified

savings to every participant, kit, and product in the survey data. Survey ex ante and survey-verified

savings are defined as follows:

• Survey ex ante savings are reported ex ante savings assigned to the subset of program

participants who returned a HEW and provided enough information to verify their savings for a

particular product.

• Survey-verified savings are savings verified by Cadmus and assigned to the subset of program

participants who returned a survey. Survey-verified savings incorporate data from responses to

HEW questions about product installations, home characteristics, and heating and water heating

fuel saturations.

Cadmus assigned survey ex ante and survey-verified savings to program participants based on the

criteria listed in Table N-1. See the Survey-Verified Savings Inputs section in this appendix for a

discussion on the Pennsylvania TRM inputs that Cadmus collected through survey data.

Table N-1. Criteria for Assigning Survey Ex Ante and Survey Verified Savings

Criteria Survey Ex Ante

Savings(1)

Survey-Verified

Savings

Whether the respondent answered the product-specific question(s) ✓ ✓

How the respondent answered questions about home characteristics ✓

How the respondent answered the questions asking if products were installed ✓

(1) Cadmus used the ICSP-reported ex ante savings for survey ex ante savings based on the listed criteria.

Table N-2 shows an example of how Cadmus assigned survey ex ante and survey-verified savings to each

program participant and kit product, which, in this example from PY8, is a showerhead from the Take

Action cohort. Cadmus included participants in the realization rate analysis if it could definitively verify

achieved savings for a particular product. Cadmus did not include participants who did not return a HEW

or did not respond to the necessary installation question in the realization rate analysis because it could

not verify savings.

In the example, Cadmus had enough information from program participants A through E to verify their

showerhead savings:

• Although Participant D did not respond to the installation question, the participant did not have

electric heat and was ineligible to receive savings regardless of whether the product was

installed. Therefore, Cadmus assigned this participant 0 kWh/yr savings.

• Similarly, although Participant E did not indicate water heat fuel type, the participant reported

not installing the product, so Cadmus assigned this participant 0 kWh/yr savings.

• Cadmus did not include Participant F’s showerhead savings in the realization rate because it

could not confirm whether the participant installed the showerhead from the kit based on the

participant’s response to the installation question in the survey.


• Cadmus did not include Participant G’s showerhead savings in the realization rate because it

could not confirm whether the participant had electric water heat based on the participant’s

response to the water heating fuel type question in the survey.

• Participants X, Y, and Z did not return a survey, so Cadmus could not verify savings for any of

their products.

Table N-2. Example of Assigning Survey Ex Ante and Survey Verified Savings

Program

Participant

(PPL EU)

Reported

Ex Ante

kWh/yr

(ICSP)

Survey

Respondent

(Survey

Data)

Installed

Product

(Survey

Data)

Electric

Water Heat

(Survey Data)

Survey

Ex Ante

kWh/yr

Survey

Verified

kWh/yr

Savings

Included in

Realization

Rate

Participant A 121.42 Yes Yes Yes 121.42 390.09(1) Yes

Participant B 121.42 Yes No Yes 121.42 0 Yes

Participant C 121.42 Yes Yes No 121.42 0 Yes

Participant D 121.42 Yes No Response No 121.42 0 Yes

Participant E 121.42 Yes No No Response 121.42 0 Yes

Participant F 121.42 Yes No Response Yes N/A N/A No

Participant G 121.42 Yes Yes No Response N/A N/A No

Participant X 121.42 No N/A Yes N/A N/A No

Participant Y 121.42 No N/A Yes N/A N/A No

Participant Z 121.42 No N/A No N/A N/A No

(1) For this example, Cadmus assumed the respondent indicated four people in the home and two showers to calculate

survey-verified savings.

Ex Post Verified Savings (Flow Diagram Items 6 & 7)

To calculate cohort-level ex post savings, Cadmus applied the cohort-level realization rates to cohort-

reported ex ante savings. Taking the sum of cohort-level ex post savings estimated the program-level

ex post savings.

Cadmus calculated confidence and precision for the ex post savings and realization rate estimates for

the Innovation Pilot cohort.

Survey-Verified Savings Inputs

Cadmus independently verified savings according to the Pennsylvania TRM and the associated

algorithms. These algorithms include open variables for which the ICSP or Cadmus can use either the

default or the option of “EDC data gathering.” Table N-3 lists the algorithm inputs, method of data

collection, and source of the data collected.


Table N-3. Pennsylvania TRM Algorithm Open Variables for Innovation Pilot Cohort

Product Open Variable Data Collection

Method Data Collector

LED ISR PY10 HEW ISR ICSP’s Subcontractor

Wattage of installed bulb Spec sheet ICSP

Showerhead

ISR PY10 HEW ISR ICSP’s Subcontractor

gpm of installed Spec sheet ICSP

Number of persons in household PY10 HEW ICSP’s Subcontractor

Number of showers in household PY10 HEW ICSP’s Subcontractor

Water heater fuel PY10 HEW ICSP’s Subcontractor

Smart Power Strip ISR PY10 HEW ISR ICSP’s Subcontractor

Use (entertainment, computer, unspecified) PY10 HEW ISR ICSP’s Subcontractor

Water Heater Setback

Number of degrees water heater turned down

(calculated using the midpoint of the ranges

provided in HEW response options)

PY9 HEW ICSP’s Subcontractor

Washing machine located in home PY10 HEW ICSP’s Subcontractor

Water heater fuel PY10 HEW ICSP’s Subcontractor

N.1.2 Database Review Findings

Cadmus reviewed both PPL Electric Utilities’ and the ICSP’s databases as well as the sources for inputs

used in ex ante savings calculations. For the Innovation Pilot, Cadmus compared the number of HEWs

the ICSP collected and the number of HEWs reported in PPL Electric Utilities’ tracking database.

The database the ICSP provided to Cadmus contained 2,038 PY10 Innovation Pilot HEWs (that Cadmus

used for its savings analysis), which aligned with PPL Electric Utilities’ tracking database.

Appendix O. Evaluation Detail – Demand Response Program PPL Electric Utilities | O-1

Evaluation Detail – Demand Response Program

O.1 Gross Impact Evaluation

This appendix describes the methodology for estimating savings and program load impacts.

O.1.1 Methodology


In PY10, 60 facilities operated by 24 customers of PPL Electric Utilities participated in one or more Act

129 demand response events. Table O-1 shows the number of participating facilities by customer

stratum. Half of the participants were small commercial and industrial (C&I) facilities, one-third were

large C&I customers, and the remaining were GNE customers. Cadmus estimated load impacts for all

participant facilities for one or more events.

Table O-1. PY10 Program Sampling Strategy

Stratum

Population

Size

(Facilities)

Target

Levels of

Confidence

& Precision

Target

Sample

Size

Achieved

Sample

Size

Evaluation Activity

Small C&I 30 N/A 30 30 Analysis of load impact data

Large C&I 20 N/A 20 20 Analysis of load impact data

GNE 10 N/A 10 10 Analysis of load impact data

Program Total 60 N/A 60 60 Analysis of load impact data


Cadmus analyzed advanced metering infrastructure (AMI) interval consumption data for each

participating facility. A facility was defined as the area over which the participating customer’s electricity

consumption was metered and the load reductions measured during PY10 Demand Response Program

period (June 1, 2018, through September 30, 2018). Cadmus estimated the facility load impacts as the

difference between baseline electricity demand and metered demand, as shown in this equation:

kW impact = Baseline kW - Metered kW

Baseline demand is a counterfactual and represents what the facility’s load would have been if the load

curtailment event had not been called. The baseline is unobservable and must be estimated. Accurate

estimation of load impacts requires establishing a valid method for estimating the baseline.

Data Collection

Cadmus collected data from several sources to evaluate the PY10 Demand Response Program impacts.

Table O-2 lists the data and sources.


Table O-2. Data Sources

Data Population Period Variables Source

Customer information

system data

Demand Response

Program participant

facilities

From beginning of

enrollment to end of

summer 2018

Customer name,

account number,

business segment,

ICSP baseline

calculation method,

enrolled MW, event

hour participation

indicators and

reported load

reductions, advance

notification times,

PJM economic market

participation dates

CPower (ICSP)

PJM day-ahead

forecasts and Act 129

event dates and hours


Demand Response

Program participants

Summer 2018 Event dates and hours PJM Interconnection

LLC website

Facility interval

consumption data


Demand Response

Program participants

April 1, 2018–

September 15, 2018

15 minute or hour

interval kWh,

estimated read

indicator


Weather 11 weather stations in


service area

April 1, 2018–

September 15, 2018

Dry-bulb temperature NOAA

Solar radiation Penn State,

Pennsylvania

SURFRAD site

April 1, 2018-

September 15, 2018

Global horizontal

irradiance

NOAA ESRL GMD

Line losses Commercial and

industrial electric

utility customers

Phase III Act 129 Line loss factor PA Technical Resource

Manual (2016), Table

1-4

PPL Electric Utilities provided 15-minute or one-hour interval consumption data between April 1, 2018,

and September 15, 2018, for 60 participating facilities. Cadmus aggregated all facility 15-minute interval

data to the hour level. A small percentage of intervals was estimated or included one or more estimated

or missing 15-minute intervals. Cadmus flagged these observations and set them to missing for the

analysis. Estimated readings were not used in the calculation of facility baselines or in estimating

savings. It was not possible to estimate demand savings of three small commercial facilities during one

or two events because the interval kWh readings for event hours were estimated and not actual

readings.

Cadmus also screened the data for outliers but did not remove any observations. A number of big box

stores had negative readings during midday hours. Cadmus inferred from the time of day and outside

temperature as well as corroborating articles in the press about solar panel installations by participating

big box store chains that these probably represented negative net demand for utility-supplied electricity

because of on-site solar generation of electricity.


Baseline Calculation Approach

“Almost Act 129 event days” were the two non-notification, non-holiday weekdays with the highest PJM

RTO day-ahead load forecasts that did not qualify as event days. These days (June 18, 2018, and August

29, 2018) had the highest day-ahead PJM forecasts that did not qualify them as Act 129 days and which

provided a natural baseline for assessing the impact of Act 129 events.186

Day-Matching Customer Baselines and Regression Baselines

Cadmus estimated individual consumption baselines for each participating facility and event using either

a day-matching approach or regression. Day-matching identifies a set of nearby, non-event, non-holiday

weekdays for each event day, referred to as the basis window. For each event hour, the baseline is the

average consumption during the same hour of the days or subset of days in the basis window. Cadmus

considered and tested the accuracy of a variety of general day-matching methods for estimating the

baselines of participating facilities:

• Y Previous Days: This is the average load of Y previous days in the CBL (customer baseline) basis

window.

• X Highest of Y Previous Days: This is the average load of the X days with highest loads of Y

previous days in the basis window.

• Y Previous Days of Same Day Type: This is the average load of Y previous days of the same day

type (e.g., Wednesday) in the basis window. For example, if Y=3 and the event occurs on a

Wednesday, the CBL basis window would only include three previous Wednesdays.

When applying a day-matching method, Cadmus excluded the following types of days from the basis

window:

• Days with average load between 2 p.m.

and 6 p.m. less than 25% of the average

load of all days in the baseline window.

This exclusion follows PJM protocol and

should result in the exclusion of most days

when a facility had abnormally low

consumption. Cadmus replaced excluded

days with the next closest permissible day.

• Weekend days

• Holidays

• Facility closures

• Previous event days187

• Weekdays more than 45 days before the

event day

• PJM economic participation days

• Act 129 notification days

Cadmus did not make any adjustments to the estimated day-matching baselines based on the difference

between the baseline and the metered load during hours preceding the event. Adjustments of this kind

186 The peak day-ahead forecasts for June 18, 2018 and August 29, 2018 were, respectively, 95.8% and 95.1% of the PJM summer peak demand.

187 Cadmus also excluded June 26, 2018, from basis windows as the ICSP informed Cadmus that PJM conducted a demand response test event.


were not permitted because PPL Electric Utilities’ Demand Response Program involved day-ahead

notification of Act 129 events.188 In the PY9 evaluation, Cadmus provides evidence that some

participating facilities appear to have adjusted their loads in response to the advance notifications.

The ICSP employed day-matching to estimate impacts and make settlement calculations. By aligning, to

the extent possible and without sacrificing accuracy, its day-matching baseline calculation methods with

ICSP’s, Cadmus eliminated a possible source of difference between the reported and evaluated impact

estimates.

Cadmus employed regression analysis as the second baseline calculation approach. Regression involves

estimating an equation to predict hourly consumption as a function of multiple independent variables

such as day of the week, hour of the day, and weather. Regression controls for the impacts of weather

on energy consumption better than day-matching and is expected to be superior to day-matching

especially for facilities with weather-sensitive loads. Cadmus estimated a separate regression model for

each facility using data for hours between 2:00 p.m. and 6:00 p.m. on the 30 non-holiday weekdays

between June 1, 2018, and September 15, 2018, with the highest day-ahead PJM RTO forecasts that did

not qualify as Act 129 event or notification days.189

Selection of Facility Baseline Calculation Methods

Before PY9 for previous Demand Response Program participants or before the start of PY10 for new

participants, Cadmus assigned each facility to one of the following day-matching baseline calculation

method or regression:

• 2 previous days

• 3 previous days

• 4 previous days

• 5 previous days

• 10 previous days

• 3 of 5 previous days with highest average

load during event hours





• 3 previous days of the same day type

(e.g., Wednesdays)

• 4 previous days of the same day type

• Regressions (one of 81 models)

188 See Goldberg, Miriam, and G. Kennedy Agnew. Measurement and Verification for Demand Response. Prepared for the National Forum on the National Action Plan on Demand Response: Measurement and Verification Working Group. 2013. The exception to this rule would be an adjustment based on an exogenous variable such as weather or the PJM day-ahead forecast of load or actual load.

189 The PJM RTO day-ahead forecast for these 30 days ranged between 82.1% and 95.8% of the PJM RTO summer peak demand forecast.


Cadmus selected the most accurate baseline calculation method for each participating facility based on

tests of predictive accuracy.190 Cadmus tested baseline calculation methods using AMI meter data from

summer 2016 for previous (PY9) participants and from summer 2017 for (PY10) participants. For

facilities assigned a regression baseline calculation method, Cadmus tested an expanded set of 81

regression models. These models included various combinations of date, time, and weather regressors

including dry-bulb-temperature (temp), cooling degree hour variables with 70°F and 75°F base

temperatures (CDH70, CDH75), a cooling degree buildup variable (CDH_buildup), temperature humidity

index (THI), and a solar radiation measure of global horizontal irradiance (GHI).191 GHI was included to

improve the predictive accuracy of regression baseline calculations for facilities with on-site solar

generation.

Table O-3 shows counts of participating facilities by final baseline modeling approach for all facilities, by

customer segment, and for 19 facilities with capacity enrollments greater than or equal to 1 MW. These

19 facilities accounted for 95% of enrolled capacity.

Table O-3. Number of Facilities by Baseline Modeling Approach

Baseline All Facilities GNE Large C&I Small C&I DR Capacity

≥ 1 MW

2 OF 2 3 0 3 0 3

3 OF 3 1 0 1 0 1

3 OF 5 1 0 1 0 1

4 OF 4 0 0 0 0 0

4 OF 5 1 0 1 0 1

5 OF 5 1 0 1 0 1

7 OF 10 6 1 5 0 5

10 OF 10 2 0 2 0 2

Day of Week 4 of 4 2 0 2 0 2

Day of Week 3 of 3 0 0 0 0 0

Regression 43 9 4 30 3

Total 60 10 20 30 19

190 Cadmus performed a separate analysis for each facility, selecting the day-matching or regression baseline method that performed best in terms of accuracy, bias, and variability (risk). It assessed the accuracy of the baseline using relative root mean squared error (RRMSE), bias using mean absolute percentage error (MAPE) and median percentage prediction error, and variability using the distribution of errors. Cadmus calculated and plotted the distribution of errors to see if there were a small number of hours where models predicted poorly.

191 The heat buildup variable was the weighted average of CDHs in the preceding 24 hours. The weights were normalized to sum to one and the weight assigned to hour t-1 was 90% of the weight assigned to hour t, so that more recent hours received greater weight.


Many large C&I facilities used day-matching approaches because they had near constant or highly

variable day-to-day consumption between 2:00 p.m. and 6:00 p.m., and regression did not predict

better than day-matching methods. For these facilities, the best predictor of consumption was

consumption in recent previous days, so many large C&I facilities selected X-of-Y-previous-day baseline

methods.

Standard Errors of Demand Savings Estimates

Cadmus calculated 90% confidence intervals for the Demand Response Program gross verified demand

savings from the standard errors for the savings estimates of individual facilities.192 For facilities with

regression baselines, Cadmus obtained the standard errors for the hourly demand savings estimates

from the regression coefficient standard errors. For facilities with day-matching baselines, Cadmus

followed the SWE’s and the PJM’s guidance to predict loads on non-event days in 2018 and to estimate

the margin of error at the 90% confidence level as the RMSE. Cadmus calculated the RMSE for the day-

matching baseline using baseline predictions for hours between 2:00 p.m. and 6:00 p.m. on non-holiday,

non-event, and non-notification days between June 1, 2018, and September 15, 2018.

O.1.2 Results and Discussion

Across the six events, PPL Electric Utilities averaged 112 MW, and averages 117 MW for Phase III event,

putting the program on track to exceed PPL Electric Utilities’ target of 92 MW for Phase III of Act 129.

PPL Electric Utilities achieved the maximum event demand savings of 120.2 MW on August 28 and the

minimum event demand savings of 102.6 MW on September 5. As Figure O-1 shows, large C&I

customers were responsible for more than 95% of the demand response savings. Unless noted

otherwise, all demand load impacts have been adjusted for line losses.

192 The standard errors for the event savings estimates do not account for the covariance of a facility’s savings across event hours, i.e., the calculation assumes the errors were independent. Calculation of event savings as the average of the event hour savings (instead of as the average of facility savings across event hours) complicates the calculation of the standard errors. However, ignoring the covariance of facility savings across event hours has little effect. Cadmus performed a separate calculation of the event savings as the average of individual facility event savings and the standard errors that account for the covariance of facility savings across event hours was only 6% larger for the July 2, 2018, event, 3% larger for the July 3, 2018, event, 4% larger for the August 6, 2018, event, 1% larger for the August 28, 2018, event, 1% larger for the September 4, 2018, event, and 1% larger for the September 5, 2018, event.


Figure O-1. PPL Electric Utilities Act 129 Gross Verified Demand Savings, PY10

Error bars show 90% confidence intervals. The Phase III demand response target for PPL Electric Utilities is 92 MW.

Appendix P. Non-Energy Benefits PPL Electric Utilities | P-1

Non-Energy Benefits

P.1 Non-Energy Benefits of Water-Saving Products

Cadmus quantified non-energy benefits in accordance with the SWE’s Guidance Memo.193 Non-energy

benefits associated with water-saving measures include the gallons of water saved. According to the

recommendation in the Guidance Memo, Cadmus assumed $0.01 in avoided cost, per-gallon saved, in

TRC testing (after gross-up for distribution losses). Cadmus assumed 20% losses on water distribution,

which is the low end of the range provided in the guidance memo (20% to 25%). The avoided cost of

water is escalated over the TRC test horizon using the same inflation/escalation assumption embedded

elsewhere in the TRC model.

P.2 Lighting Interactive Effects

Cadmus calculated lighting interactive effects according to the Guidance Memo. The memo states

“Installation of LED lighting equipment in homes and businesses with natural gas heating systems leads

to an increase in gas usage because LEDs generate less waste heat than inefficient technologies. The

reduced heat in the space must be compensated for by the heating system. The PA TRM provides

interactive effect assumptions for electric heating and cooling systems, but not fossil fuel… The gas

heating fuel share and percentage of lamps installed in interior sockets are taken from the 2014

Residential Baseline Study (Tables 5-29 and 5-50 and Figure 5-12).”

P.3 Non-Energy Benefits of Natural Gas Savings

Per the Guidance Memo, Cadmus assumed that there is a natural gas therms penalty (negative benefit).

Cadmus applied the therms penalty to the ex post kWh/yr savings, which incorporates the electric

energy heating penalty in accordance with the TRM.

Cadmus calculated therm benefits using the average annual avoided gas costs submitted with PPL

Electric Utilities’ Phase III EE&C plan.194 A distribution loss factor was applied to gross up impacts in the

home to the water heating system.

Cadmus developed values for the non-residential programs from the 2014 PA C&I baseline study195 (as

shown in Table P-1) for the assumptions needed to compute the heating penalty in commercial

buildings.

193 SWE. Guidance on the Inclusion of Fossil Fuel and H2O Benefits in the TRC Test. March 25, 2018.

194 PPL Electric Utilities’ revised EE&C plan (Docket No. 2015-2515642) filed with the PA PUC November, 2018.

195 Nexant, Inc. (with GDS Associates, Research Into Action, and Apex Analytics). Pennsylvania Statewide Act 129 2014 Non‐Residential End Use & Saturation Study. Prepared for the PA PUC. April 4, 2014. Available online: http://www.puc.state.pa.us/Electric/pdf/Act129/SWE-2014_PA_Statewide_Act129_Non-Residential_EndUse_Saturation_Study.pdf

http://www.puc.state.pa.us/Electric/pdf/Act129/SWE-2014_PA_Statewide_Act129_Non-Residential_EndUse_Saturation_Study.pdf

http://www.puc.state.pa.us/Electric/pdf/Act129/SWE-2014_PA_Statewide_Act129_Non-Residential_EndUse_Saturation_Study.pdf

Appendix P. Non-Energy Benefits PPL Electric Utilities | P-2

Table P-1. Non-Residential End-Use Penetration and Fuel Shares

End Use Penetration Fuel Share

Electric Natural Gas Fuel Oil Other(1) n-values(2)

Lighting 100.0% 100.0% 0.0% 0.0% 0.0% -

Space Heating(3) 100.0% 6.8% 84.4% 4.3% 4.5% 449

Space Cooling 84.3% 100.0% 0.0% 0.0% 0.0% -

Plug Load 100.0% 100.0% 0.0% 0.0% 0.0% -

Refrigeration 35.0% 100.0% 0.0% 0.0% 0.0% -

Cooking 27.9% 53.3% 42.5% 0.0% 4.2% 659

Water Heating(3) 92.7% 37.8% 56.3% 1.9% 3.8% 540

Other(4) 100.0% 100.0% 0.0% 0.0% 0.0% - (1) “Other” fuel share includes LPG, purchase HW or steam, wood, and misc. fuels. (2) n-values for fuel share only. (3) Fuel shares for space heating and water heating are based on square footage served and tank capacity, respectively. All others are per premise. (4) “Other” end use includes pumps, motors, and misc. equipment.

Appendix Q. Net Impact Evaluation PPL Electric Utilities | Q-1

Net Impact Evaluation

Q.1 Self-Report Survey Methodology

Q.2 Free ridership

Free ridership is a measure of the savings that participants would have achieved on their own in the

absence of the program; these savings are subtracted from verified gross savings. Spillover, on the other

hand, credits additional savings that participants achieved on their own, where their experience with the

program was highly influential in their decision to install energy-efficient equipment without the

incentive of rebates. Spillover increases net savings attributable to PPL Electric Utilities.

Following methods defined in the Phase III Evaluation Framework,196 Cadmus assessed free ridership.

This assessment involves two components—the intention to implement an energy-efficient project

without a rebate and the influence of the program in the decision to implement the energy-efficient

project. When scored, each component has a value ranging from zero to 50 and a combined total free

ridership score ranging from zero to 100.

Cadmus summed the intention and influence components to estimate the total intention/influence

method free ridership average by product or stratum. Nonresidential scores are weighted by ex post

gross kWh/yr savings.

Intention Score

Cadmus assessed intention by asking questions to determine how the participant’s decisions would have

differed in the absence of the program. For example, surveys asked the following key questions to

determine how the residential respondent’s, or the business organization’s project-related decisions,

would have differed in the absence of a program:

• “Which of the following would have happened if you had not received the rebate for $[REBATE

AMOUNT] from PPL Electric Utilities for the [MEASURE OR C_MEASURE] project?”

• “By how much would you have reduced the size, scope, or efficiency?”

• “How likely is it that [you/your organization] would have paid the full cost to install the same

quantity and efficiency of that equipment at the same time you conducted this project?”

Cadmus used the responses to determine a participant’s final intention score, which was multiplied by

the participant’s respective ex post kWh/yr savings to calculate intention-based free rider savings.


Appendix Q. Net Impact Evaluation PPL Electric Utilities | Q-2

Influence Score

Influence is assessed by asking about how much influence—from 1 (no influence) to 5 (extremely

influential)—various program elements had on the customer’s decision to purchase energy-efficient

equipment. The survey asked the following influence question:

• “Please rate each item on how much influence it had on the decision to complete the project

the way it was completed. Please use a scale from 1, meaning no influence, to 5, meaning the

item was extremely influential in your decisions.”

From responses to this question, Cadmus obtained data about the influence of various program

components. Cadmus assessed influence from participants’ ratings of how important various program

elements were in their decision to purchase energy-efficient equipment.

Q.3 Spillover

Following methods defined in the Phase III Evaluation Framework,197 Cadmus estimated spillover. To

estimate spillover, surveys included questions to determine whether participants installed specific

additional high-efficiency products and, if so, whether participation in the program was important to

their decision. Additional high-efficiency product purchases counted only toward spillover if the

customer did not receive a rebate and the program had been important to the decision to purchase and

install the products. Typically, the data collected through the surveys does not provide enough

information to reliably quantify spillover; therefore, spillover is reported qualitatively.


Appendix R. Survey Methodology PPL Electric Utilities | R-1

Survey Methodology

In presenting interview and survey data in the report, the percentage or frequency of responses is

followed by the sample size for the particular question. Sample size (denoted by “n”) refers to the

number of respondents who answered the question. Sample sizes may vary by question, because of

survey logic and skipped questions. Respondents could skip questions if they did not want to answer

them; not all respondents provided an answer to every question.

R.1 Survey Bias

Surveys employ the self-report method, which can result in validity issues and biases (e.g., self-selection,

recall, social desirability). Cadmus designed the surveys to minimize such issues and biases using these

best practices:

• Avoid questions that are leading, ambiguous, or contain more than one topic

• Employ randomization of list-based survey items to reduce order effects

• Use consistent survey wording and response options for online and phone surveys when

relevant

• Employ stratified random sampling when relevant

The SWE team and PPL Electric Utilities reviewed and approved new surveys that Cadmus fielded in

PY10.

R.2 Survey Contact Instructions

Cadmus coordinated with PPL Electric Utilities’ contractor to screen the sample and remove the records

of any customers called in the past three months (whether for a Cadmus survey or a PPL Electric Utilities

survey), had requested not to be contacted again, or had incomplete information. Cadmus also excluded

inactive customers and customers who were selected for another survey. This cleaning and survey

sample preparation process reduced the available sample.

For online surveys, Cadmus sent email invitations to the remaining contacts with email addresses and

followed up with one reminder email invitation. For telephone surveys, Cadmus attempted each record

up to five times at different times of the day and weekend and left messages with voice mail where

possible. For multi-mode surveys, Cadmus first contacted all participants with email addresses to

complete an online survey, sent one reminder email invitation and then telephoned participants who

did not have a valid email address or did not respond to the online survey. Giving participants two

avenues to respond to the survey increased response rates in programs with limited population.

Documents

PPL Electric Utilities - PA.Gov · 2019. 11. 26. · PPL Electric Utilities Annual Report to the Pennsylvania Public Utility Commission PHASE III OF ACT 129 PY10 ANNUAL REPORT (JUNE