3. LIEE Cost-Effectiveness Results

In the following sections we briefly summarize the LIEE cost-effectiveness methodology and results presented by the Standardization Team in their final report, followed by our consideration of the Team's recommendations.

3.1. Overview of Methodology

The methodology adopted in D.02-08-034 considers the cost-effectiveness of the LIEE program and measures from two perspectives: cost efficiency from the perspective of the non-participant, and hardship reductions from the perspective of the participant. To this end, we apply two tests of cost-effectiveness: a modified Participant Cost (PC_m) Test and a Utility Cost (UC) Test. The PC_m Test divides the participant benefits by the utility costs of the program or measure.¹⁹ This test produces a benefit-cost ratio that expresses the level of the participants' benefits, relative to program dollars. The UC Test produces a ratio of the benefits to the general ratepayers who subsidize the program, relative to program costs. Both tests are designed to incorporate a set of non-energy benefits (NEBs) as well as direct energy-related benefits. These NEBs are meant to capture a variety of effects such as changes in comfort and reduction in hardship, which are not captured by the energy savings estimates derived from a load impact billing evaluation and are ignored in more traditional cost effectiveness approaches. The Commission adopted the methods for calculating NEBs in D.02-08-034.

In assessing overall program cost effectiveness, the Standardization Team considered both direct measure costs and a variety of indirect costs (administration costs, outreach, shareholder earnings, etc.). In evaluating the cost effectiveness of individual measures, however, only installed measure costs were considered. As explained in the utilities' September 30, 2002 filing, the rationale for this latter approach is that, from an economic perspective, cost effectiveness analysis should consider only those costs that are truly affected by the decision at hand. These are sometimes called incremental costs, or marginal costs. In applying the cost effectiveness framework to individual measures, the decision at hand is whether or not a specific measure should be added to or dropped from the program. Insofar as retaining or dropping a specific measure will have a relatively minor impact on indirect costs, these indirect costs were ignored in the application of the measure level cost effectiveness tests.

For all measures, cost effectiveness ratios were developed by residence type and (where applicable) fuel type. For measures with weather-sensitive effects, the analysis was also conducted for individual climate zones. The climate zones used for this purpose were the California Energy Commission's sixteen Title 24 climate zones, which are depicted in Figure 1. This disaggregated approach was designed to recognize the variation in benefits and costs across specific applications of the measures in question. However, it also yielded situations in which measures were cost-effective in some applications (some residence types, some climate zones, or one fuel) but not others. In the September 30, 2002 preliminary report, the Standardization Team made recommendations for the treatment of these situations on a case-by-case basis. In its subsequent comments, ORA objected to the nonsystemmatic nature of these preliminary recommendations and proposed that the Team develop more systematic decision rules to be used to maintain consistency in the treatment of these cases. The Team developed such rules and presented them in the June 2, 2003 final report, as discussed further below.

In the analysis underlying its final report, the Team used per measure savings estimates based on the recent load impact evaluation of the 2001 LIEE Program.²⁰ It reflects the Team's judgment that the measure-specific impacts provided in the 2001 evaluation are superior to those developed in the 2000 load impact study. This judgment is based in turn largely on the specific design of the 2001 impact evaluation. In previous impact evaluations, the primary focus had been on the estimation of overall program savings, although savings were developed for individual measures and groups of measures. In response to the Commission's instruction to the joint utilities to assess cost-effectiveness of individual measures and to use these results in measure selection, the Team requested that the project consultant, XENERGY, refine the 2001 impact analysis to more effectively isolate individual measure impacts. This refinement took the form of an extensive review and revision of the preliminary engineering estimates used in the development of weights for measure savings in the XENERGY billing analysis model. Many of these engineering estimates were derived from the Database for Energy Efficiency Resources (the DEER database), which was developed by XENERGY under a previous statewide project.²¹ In addition, XENERGY refined the analysis to better isolate the savings from ceiling insulation by estimating a separate statistical adjustment coefficient for that measure.

While the Team considers the 2001 impact study estimates the best available estimates for the purposes of cost-effectiveness assessment, it cautions that all estimates are subject to statistical error. Estimates of savings from measures with low impacts are particularly subject to high percentage errors as a result of inherent difficulties in isolating these impacts in the statistical analysis of changes in energy consumption. The Team also notes that subsequent years' program impact evaluation studies may yield measure savings estimates that differ somewhat from those used in this study, and that reconsideration of the program measure mix may be necessary over time as such changes occur.

3.2. Team Recommendations

In D.02-08-034, the Commission adopted a set of guidelines for considering whether to retain specific LIEE program measures. First, measures that have both a PC_m and UC benefit-cost ratio greater than or equal to the average program PC_m and UC should be included in the LIEE program. This applies for both existing and newly proposed measures. This approach encourages improvement in program efficiency by selecting measures that will improve the cost-effectiveness of the LIEE program from both the participant and non-participant perspectives.

Second, existing measures with one of the two benefit-cost ratios less than the average program PC_m and UC for that utility would be retained in the program. However, new measures meeting this criterion would not be accepted because of the substantial effort required to integrate a new measure into the program.

Finally, existing and new measures with both the UC and PC_m test results less than the average program PC_m and UC for that utility should be excluded from the LIEE program unless substantial argument can be made that significant NEBs are not currently being accounted for in the PC_m and UC test values or there are other policy or program considerations that require the measure to be retained.

Attachment 2 presents the detailed results of the Team's cost-effectiveness evaluation, in tabular form. We summarize the results and the Team's recommendations in the following sections. We refer to measures with benefit-cost ratios that exceed the program average test results as "passing" the PC_m or UC tests, based on the adopted guidelines described above.

3.2.1. Non-Weather Sensitive Measures

Non-weather sensitive measures are those whose impacts do not vary across climate zones. These include hard-wired compact fluorescent porch lights, compact fluorescent lamps (CFLs), faucet aerators, low-flow showerheads, high efficiency refrigerators, water heater blankets, water heater pipe wrap, and high-efficiency water heaters. Based upon the application of the cost effectiveness criteria and judgments with respect to other factors, the Standardization Team presented the following recommendations with respect to individual non-weather sensitive program measures:

· Hard-wired compact fluorescent porch lights for single-family applications pass the PC_m and UC tests in PG&E and SDG&E's service areas, and nearly satisfy both tests for SCE. However, this measure fails both tests in PG&E's and SCE's service areas for multifamily residences and mobile homes by a fairly large margin. The Team recommends that this measure continue to be offered for single-family homes but not for multifamily residences or mobile homes.

· CFLs pass both tests in all applications, and should continue to be offered for all residence types and in all climate zones.

· Faucet aerators pass the PC_m Test and/or the UC Test in all applications and should continue to be offered for all residence types and in all climate zones.

· Low-flow showerheads are cost-effective in all applications, and should continue to be offered for all residence types and in all climate zones.

· High efficiency refrigerators pass both tests in all applications, and should continue to be offered for all residence types and in all climate zones.

· Water heater blankets pass both tests in all applications, and should continue to be offered for all residence types and in all climate zones.

· Water heater pipe wrap passes both tests in all applications, and should continue to be offered for all residence types and in all climate zones.

· High efficiency water heaters do not pass either test for any water heating fuel, residence type, or service area. As a result, the Team recommends that they be dropped from the program.

3.2.2. Weather-Sensitive Measures

Weather-sensitive measures are those whose impacts vary significantly across climate zones. While the Commission mandated only that ceiling insulation be evaluated at the climate zone level, the Team agreed to assess all weather-sensitive measures (other than outlet gaskets) at this level. For these measures, the Team faced three options: offer a measure in all climate zones; do not offer the measure in any climate zone; or offer the measure in selected climate zones. For the purposes of the analysis, the Team further divided weather-sensitive measures into two groups; infiltration reduction measures and non-infiltrationreduction measures.

3.2.2.1. Infiltration-Reduction Measures

Infiltration-reduction measures are measures whose primary effect is to reduce air transfer through the thermal shell in participating homes. The LIEE program currently includes five such measures: caulking, door weatherstripping, attic access weatherstripping, evaporative cooler covers and outlet gaskets. The cost-effectiveness of these five measures is discussed below.

· Caulking. Caulking fails both the PC_m and the UC test in all applications for three of the utilities. However, caulking passes one or both tests in some or all climate zones for SoCalGas. The reason for this difference is that SoCalGas reports far lower installed costs for caulking than the other utilities.

· Evaporative cooler covers. Evaporative cooler covers are not cost effective in multifamily dwellings or in any residence type with electric space heat. However, they are cost effective for SDG&E in two climate zones for homes with gas heat.

· Weatherstripping attic doors. Weatherstripping attic doors does not appear to be cost effective in any zone, for any heating fuel, or in any residence type.

· Weatherstripping doors. Door weatherstripping does not appear to be cost effective for any residence types, heating system, or climate zone

· Outlet gaskets. Measure impact estimates were not available by climate zone for this measure, in spite of the fact that its impacts are weather-sensitive. As a result, an overall average cost-effectiveness ratio was developed for each heating fuel and each residence type. Outlet gaskets pass the PC_m and/or the UC test when electric space heat is present in a majority of cases, but fail both tests for all applications involving gas space heating.

The cost-effectiveness results for these measures are not particularly favorable. Nonetheless, the Team recommends that they be retained for the 2004 LIEE program. The Team bases this recommendation on a number of factors:

· In general, the cost of installing these measures is quite low, and the energy savings are correspondingly low. The Team notes that the degree of uncertainty surrounding the estimates of energy savings for these measures is relatively high, and it is extremely difficult to isolate their impacts on energy consumption through engineering analysis, billing analysis, or a mix of the two (as in the PY2001 impact evaluation).

· There may be significant interactions between infiltration-reduction measures and other weatherization measures. That is, infiltration reduction measures may enhance the savings from other measures through thermodynamic interactions.

· Infiltration-reduction measures lower draftiness and thereby provide significant non-energy benefits relating to comfort. While comfort benefits are included in the NEB workbook, they are allocated across a wide range of measures encompassing both infiltration-reduction and non-infiltration-reduction measures. The Team believes that the allocation of these benefits by energy savings probably understates the benefits associated with infiltration-reduction measures.

· Most of these measures tend to be installed in a high percentage of participating homes. As pointed out by the ICA in workshop comments, their costs may be overstated if contractors disproportionately assign "windshield drive time costs" to these measures.²² Dropping these measures could have the impact of adversely affecting the costs (and cost effectiveness) of other measures in the future, should contractors assign more of their indirect costs to those measures.

· Eliminating the measures would significantly reduce the number of homes weatherized.

3.2.2.2. Non-Infiltration Reduction Measures

Non-infiltration reduction measures are measures for which the primary effect on energy use is through some mechanism other than infiltration reduction. Several LIEE measures fall into this category. Results and recommendations relating to these measures are presented below.

· High efficiency central air conditioner replacements. High efficiency air conditioners do not pass either the PC_m or UC Test in any case except for one utility (SDG&E) in Climate Zone 15. The Team recommends that this measure be dropped from the program in all areas.

· High efficiency room (window/wall) air conditioner replacements. High efficiency room air conditioners are cost effective in climate zones 13 and 15 for one of the utilities serving those zones. Nonetheless, the Team recommends that high efficiency room air conditioners be offered for all residence types in climate zones 11-15, which have the most extreme summer conditions. The Team believes that the potential reduction in risks to customer health and safety associated with the availability of high efficiency units, which may not be fully reflected in current NEBs used in the analysis, justifies offering this measure in these extreme climate zones.

· Ceiling insulation. Ceiling insulation passes the PC_m and/or the UC Test in virtually all cases. The Team recommends that ceiling insulation be retained in the program using current polices with respect to ceiling insulation thresholds and final levels.

· Duct testing and sealing. Duct testing and sealing is cost effective only for SoCalGas and only in Climate Zone 14. The Team recommends that this measure be dropped from the program.

· Evaporative cooler maintenance. This measure is cost-effective for only one utility (SDG&E) and in only one climate zone. The Team recommends dropping this measure altogether from the program.

· Evaporative coolers. This measure is cost-effective for mobile homes and single family homes in at least one utility in climate zones 11, 12, 13, 15 and 16.²³ However, the measure is cost-effective for only one utility in one zone for multifamily dwellings. The Team recommends that evaporative coolers be retained in climate zones 11-16, but only for single family homes and mobile homes.

· Furnace filters. Furnace filters were assessed under two scenarios: (1) one where their installation either does not require a licensed Heating, Ventilation and Air Conditioning (HVAC) contractor or a licensed HVAC contractor is already on site to do a furnace repair or replacement; and (2) where a licensed HVAC contractor has to make a special trip to install the filter. Installed costs are obviously higher under the second scenario than the first. Furnace filters are cost effective in many zones and for most utilities under the first scenario. However, furnace filters are cost effective only for SoCalGas in climate zone 14 under the second (more expensive) scenario. The Team believes that it is prudent to require that furnace filters be installed by licensed HVAC contractors. As a result, the Team recommends that furnace filters be installed in all zones, but only as part of furnace repairs or replacements. To some extent, this is a practical matter, in that it would make little sense to make significant furnace repairs or to replace a furnace without replacing the filter.

· Gas Furnace Repairs. Gas furnace repair is cost-effective in some but not all zones, and for some but not all utilities. However, the Team believes that the NEBs incorporated into this analysis do not fully reflect the non-energy benefits associated with this measure. NEBs are distributed in proportion to energy savings, and the energy savings associated with this measure are assumed to be experienced only by households who were previously using their furnace. However, households who were not using their furnaces prior to repairs clearly obtain some comfort benefits as well, and these benefits are not explicitly encompassed by the method of allocating NEBs to individual measures. Moreover, there may be some safety benefits that are not recognized by the NEB framework.²⁴ In recognition of this shortcoming in the cost effectiveness methodology, the Team recommends that gas furnace repairs continue to be offered in all climate zones.

· Gas furnace replacements. Gas furnace replacements are also cost effective in only some zones and for some utilities. Using the same rationale as discussed for furnace repairs, the Team recommends that furnace replacements continue to be offered in all zones.

· Minor Home Repairs. Minor home repairs carry very significant participant NEBs, and are consequently highly cost effective according to the PC_m Test in nearly all climate zones. They are not cost effective in quite so many zones under the UC Test, largely due to the differences in participant and utility NEBs. The Team recommends that they continue to be offered in all zones, even those in which they do not appear to be cost effective. The rationale here is that these repairs are often necessary to accommodate the installation of other cost-effective measures offered through the program, and thus have an additional indirect benefit.

· Setback Thermostats. Like furnace filters, setback thermostats were assessed under two scenarios: (1) that an HVAC contractor is already on site to do a furnace repair or replacement; and (2) that an HVAC contractor has to make a special trip to install the programmable thermostat. This measure is cost effective in some climate zones for at least one utility under the first scenario. However, it fails to be cost effective in all climate zones, fuels, utilities, and residence types under the second scenario. The Team recommends that this measure be dropped from the program, except in cases where furnace repairs or replacements are being made and local code requires programmable thermostats.

· Whole house fans. Whole house fans are cost effective only in climate zone 10 for SDG&E. The Team recommends that whole house fans be dropped from the program.

3.2.3. Summary

Table 1 below presents an overview of the recommendations of the Standardization Team.

Table 1: Recommendations on Individual Measures

Measure	Recommendation
Non-Weather-Sensitive Measures
Hard-wired CFL porch lights	Retain in all climate zones for single family homes, but drop for multi-family and mobile homes
Compact fluorescent lamps	Retain in all climate zones and residence types
Faucet aerators,	Retain in all climate zones and residence types
Low-flow showerheads,	Retain in all climate zones and residence types
High efficiency refrigerators	Retain in all climate zones and residence types
Water heater blankets	Retain in all climate zones and residence types
Water heater pipe wrap	Retain in all climate zones and residence types
High-efficiency water heaters	Drop from Program
Weather-Sensitive Measures
Outlet gaskets	Retain in all climate zones and residence types
High efficiency central Acs	Drop in all climate zones and residence types
High efficiency room Acs	Retain in Climate Zones 11, 12, 13, 14, and 15
Caulking	Retain in all climate zones and residence types
Ceiling Insulation	Retain in all climate zones and residence types
Duct testing and sealing	Drop in all climate zones and residence types
Evaporative cooler covers	Retain in all climate zones and residence types
Evaporative cooler maintenance	Drop in all climate zones and residence types
Evaporative coolers	Retain in Climate Zones 11 - 16 for single family and mobile homes; drop from Program for multi-family homes and in Climate Zones other than 11 - 16.
Furnace filters	Retain, but only as part of furnace repair or replacement
Gas furnace repairs	Retain in all climate zones and residence types
Gas furnace replacements	Retain in all climate zones and residence types
Minor home repairs	Retain in all climate zones and residence types
Setback Thermostats	Drop from Program except where required by code in conjunction with furnace repair or replacement
Weatherstripping attic doors	Retain in all climate zones and residence types
Weatherstripping doors	Retain in all climate zones and residence types
Whole house fans	Drop from Program

¹⁹ As explained in D.02-08-034, the traditional Participant Cost Test ratio would be an undefined number for LIEE programs because the out-of-pocket costs to participants are generally zero. Therefore, this modified version of the test was developed and adopted for application to LIEE activities. ²⁰ See Final Report by XENERGY, Inc.: "Impact Evaluation of the 2001 Statewide Low-Income Energy Efficiency (LIEE) Program," April 2003. ²¹ XENERGY, Inc. "2001 DEER Update Study, Final Report." August 2001. The DEER database is housed at the California Energy Commission, and the Public Utilities Commission funds the research. The database represents a current standard source for estimates of incremental measure costs and engineering estimates of per-unit energy savings. It is updated periodically and available over the Internet at http://www.energy.ca.gov/forecasting/DEER.html. ²² Windshield drive time costs refer to the costs of having a crew available and moving it from one home to another. ²³ Under current LIEE Statewide Policies and Procedures, evaporative coolers are offered in climate zones 2, 3, 4, 5, 9, 10, 11, 12, 13, 14, 15, and 16. ²⁴ The NEB study initially identified CO testing as a service that may yield safety benefits, but this non-energy benefit was not estimated as part of that study. Although such safety benefits may also be associated with improved furnace operation, the NEB study did not identify them.