V. Cost-Effectiveness Testing

Pub. Util. Code § 2790(a) directs the Commission to consider "both the cost effectiveness of the services and the policy of reducing the hardships facing low-income households" in designing LIEE programs. To formalize these considerations in program evaluation, we directed the Working Group to develop cost-effectiveness testing procedures that consider reductions in hardship. The Project Team was assigned the task during Phase 3 to recommend a specific methodology for evaluating program measures using these procedures.⁵⁸

Currently, we report the cost-effectiveness of LIEE programs using three specific tests, each reflecting a different perspective. These tests originated as part of a Standard Practice Manual developed in the 1980s to evaluate demand-side management (e.g., energy efficiency) programs in general. Per D.01-03-028, we authorized the continued use of these tests for LIEE programs while the Working Group developed its Phase 2 recommendations.

The Participant Cost Test (PC) measures benefits and costs from the perspective of the customer receiving the measures or services. This test compares the reduction in the customer's utility bill, plus any incentive paid by the utility, with the customer's out-of-pocket expenses. In the case of LIEE program measures, where there generally are no out-of-pocket expenses to the eligible customer, the PC basically measures the bill savings associated with the program or measure.⁵⁹

The Utility Cost Test (UC) measures the net change in a utility's revenue requirements resulting from the program. The benefits for this test are the avoided supply costs of energy and demand ("avoided costs")-the reduction in transmission, distribution, generation and capacity costs valued at marginal cost-for the periods when there is a load reduction. The costs for the UC test are the program costs incurred by the utility, including any financial incentives paid to the customers, and the increased supply costs for the periods in which load is increased. Since this test is designed to focus on utility revenue requirements, it does not include any net costs incurred by program participants (which, in the case of LIEE programs, is usually zero).

The Total Resource Cost (TRC) test measures the net costs of a program as a resource option based on total costs, including both the participants' and the utility's costs. The benefits are calculated in the same manner as the UC test described above. The costs in this test are the total equipment or measures costs, including installation, operation, and maintenance and administration, no matter who pays for them. In addition, costs for this test include the increase in supply costs for the periods in which load is increased. When there are no co-payments or other out-of-pocket expenses required by the program participant, the TRC and UC tests are identical.

As currently applied, these tests do not include any non-energy benefits associated with the LIEE program. For example, the PC test does not include the benefit of improved comfort from weatherization. The UC test does not include savings from reduced bad debt write-offs or other impacts that saves the utility (and ratepayers) costs. The Working Group considered a number of these benefits, and quantified many of them for inclusion in cost-effectiveness testing. (See Attachment 4.) The Working Group developed adders for each of these benefits so that they could be used in conjunction with the current cost-effectiveness tests, or integrated into a single test.

In its report, the Working Group recommends that the Commission use a new, multi-perspective test, called the Low Income Public Purpose Test (LIPPT), to evaluate LIEE programs overall, as well as specific program measures. The LIPPT is designed to enhance the existing TRC by including a wide-range of non-energy benefits valued from all three perspectives. Like the TRC, the calculation of benefits for the LIPPT reflects avoided costs from a resource perspective. However, the avoided costs used in the LIPPT (unlike the traditional TRC test) also include adders for environmental externalities.⁶⁰ In addition, the LIPPT includes the utility, participant and societal non-energy benefits described in Attachment 4.⁶¹

In order to demonstrate the applicability of the LIPPT to measure assessment, the Project Team applied this test to the eight new rapid deployment measures adopted in D.01-05-033.⁶² For each measure, the LIPPT is calculated by summing the lifetime energy savings and non-energy benefits, and dividing by the measure cost. Four sets of test results are presented for each measure, by residence type and climate zone:

· LIPPT with non-energy benefits using gross savings and costs,

· LIPPT with non-energy benefits using incremental savings and costs,

· LIPPT without non-energy benefits using gross savings and costs, and

· LIPPT without non-energy benefits using incremental savings and costs.

As explained in the report, using "gross" savings and costs assumes that the old equipment would not have been replaced for some number of years at least as great as the lifetime of the new equipment. The "incremental" approach assumes that the measures would have been replaced with standard efficiency new units in the absence of the installation of high efficiency units. By presenting the results of both approaches, the Project Team illustrates the potential range of results for each measure, depending upon which replacement assumption is used. The Project Team also presents the LIPPT ratios with and without non-energy benefits to illustrate their importance in the analysis of measure cost-effectiveness. However, the Project Team did not reach any conclusions at this time on which measures should continue beyond the pilot stage, or present a discussion of how they would make such a determination using the four sets of LIPPT results.

SESCO and ICA argue that the LIPPT is flawed because the energy savings are based on the resource benefits of avoided costs, rather than the bill savings to the low-income customer. In ICA's view, the reduction in billing costs is the only quantifiable legal purpose of the LIEE program, and using avoided costs inappropriately underestimates the benefits of the program. In addition, SESCO argues against using gross savings and costs because they overstate the benefits of appliances.⁶³

We agree with SESCO and ICA that bill savings should not be ignored in the consideration of program measures or overall program effectiveness. However, we do not agree that this should be accomplished by substituting avoided costs with those savings in the LIPPT test. The fundamental problem with the LIPPT test is that, by attempting to merge the benefits and costs from several perspectives, it ends up being somewhat of a meaningless hybrid test. In our view, the LIEE program should be examined from two different perspectives, with some weighing and judgment applied to the results in selecting eligible measures or in evaluating overall program effectiveness.

The first perspective is that of the low-income customer, in terms of reducing hardship. This includes bill savings, as well as non-energy benefits that the program or measure provides to the recipient. When augmented with these non-energy benefits, the PC test provides this perspective. Since the low-income customer generally incurs no out-of-pocket expenses (making the cost component of the test essentially zero), applying the PC test to LIEE programs or measures produces a relative ranking based on hardship benefits to the participating customer.

Our evaluation of the program or individual measures cannot end with simply maximizing the hardship benefits to low-income customers. As previously stated in D.00-07-020, cost efficiency is to be evaluated and considered as well:

"...we should strive to maximize the participation of eligible participants and work to reduce their electric and gas bills as much as possible, within the constraint of limited funding. At the same time, to protect the interests of non-participating ratepayers that subsidize the costs of the program, we need to ensure that service delivery is as efficient as possible."

"Meeting the needs of low-income customers as cost-efficiently as possible is also the stated intent of the Legislature, as articulated in Pub. Util. Code §2790, recently amended by AB 1393. This section directs the utilities to meet the need for weatherization services by low-income utility customers `taking into consideration both the cost-effectiveness of the services and the policy of reducing the hardships facing low-income customers.' Consistent with that intent, we have defined the program in our DSM rules as serving `an equity objective in assisting customers who are highly unlikely or unable to participate in other residential programs' and therefore the program is not subject to strict cost-effectiveness requirements. At the same time, we have promoted the consideration of cost-efficiency in the provision of these services."⁶⁴

Therefore, we need to also evaluate the LIEE program and individual measures from a cost-efficiency perspective, in terms of the resources required to provide services to low-income customers. Only the UC test is designed to examine cost-efficiency from the perspective of those customers who directly subsidize the program costs through their rates, i.e., non-participating customers. As discussed above, the cost side of the equation is virtually identical under the TRC and UC tests, as is the calculation of energy-related benefits (avoided costs). The benefits side of this test should be enhanced to include reduced carrying costs on arrearages, lower bad debt written off, fewer notices and collection costs, and the other non-energy benefits that reduce utility revenue requirements. (See Attachment 4.)

In sum, the utilities should evaluate the LIEE program and individual measures by calculating both the PC and UC tests, as modified to include the non-energy related benefits associated with each perspective. We must then consider how the results of each test should be used in making final measures selections, or in evaluating the effectiveness of LIEE programs from year-to-year or across utilities. We must consider whether there should be a pre-determined method (e.g., weighting) of the tests established at the outset, or whether the process should involve more case-by-case judgments of test results, or whether other approaches should be used.

We will refer these issues back to the Working Group and Standardization Project Team for further evaluation and recommendations. In considering them, the utilities and interested parties should review our discussion in D.92-09-080, where we articulated our concerns over certain cost-effectiveness weighting procedures proposed by the parties.⁶⁵

In addition, the Working Group and Project Team should consider developing a more explicit method for addressing the "gross" versus "net" costs and savings issue in measure and program evaluation. SESCO's suggestion of utilizing a mix of these two approaches based on the expected life and the minimum age for replacement should be considered.

The RRM Working Group and Standardization Project Team should jointly file and serve recommendations on these issues, after obtaining public input, within 120 days from the effective date of this decision. The report should include a discussion of the pros and cons of the various options considered. Comments on the report are due 30 days after the date of filing, and replies are due 20 days thereafter. After our consideration and resolution of these issues, the Standardization Project Team should assess all current (including pilot) LIEE program measures, using the approved cost-effectiveness testing procedures. The Assigned Commissioner shall establish the budget, schedule and scope of work for this effort.