9. Standard Practice Manual-Related Anomalies
The Standard Practice Manual (SPM) contains the Commission's methodology for evaluating energy efficiency investments using various tests of cost-effectiveness.59 The policy rules adopted by D.05-04-051 (Rules) require that the energy efficiency portfolios as a whole pass both the total resource cost (TRC) test and the program administrator cost (PAC) tests of cost-effectiveness contained in the SPM. Individual program selections are also made by program administrators based on the results of these two cost-effectiveness tests, as well as other evaluation criteria. In addition, per D.05-04-051, the performance basis for the risk/reward incentive mechanism being developed in R.06-04-010 will be based on these two tests, for the majority of energy efficiency activities.
The Rules describe the TRC test as the measurement of net resource benefits from the perspective of all ratepayers produced "by combining the net benefits of the programs to participants and non-participants." The benefits are the costs of the supply-side resources avoided or deferred. The TRC costs encompass the costs of the measures/equipment installed and the costs incurred by the program administrator.60 Under the PAC test, the program benefits are the same as the TRC test, but costs are defined differently to include the costs incurred by the program administrator (including financial incentives or rebates paid to participants). The PAC test does not include the costs incurred by the participating customer.61
Three anomalies were identified in the E3 calculators used for the utility June 1, 2005 filings with respect to the SPM tests, as discussed further below.
9.1. Treatment of Load Increases
The SPM states that load increases should be treated as a cost in the calculation of the TRC and PAC net benefits (or benefit-cost ratios). The E3 calculator treats a load reduction as a positive benefit, and a load increase as a negative benefit.
As pointed out in the Final Report, this treatment does not affect the calculation of net benefits under the SPM tests, and would not affect the benefit-cost ratios to any significant degree (i.e., it would not make a program with a benefit-cost ratio greater than one have a ration less than one, or vice versa). We concur with the consensus position of the workshop participants that that this is a minor inconsistency and does not merit changes to the E3 calculator
9.2. Overhead Double Counting
The reporting requirements developed by Energy Division require that utilities report program overhead costs as part of the administrative cost category, even in cases where contractors are performing the installation work.62 The workshop discussion and parties' comments indicate that this could lead to a double counting of overhead costs in the SPM tests, because some of those costs may already be included in the labor component of the incremental measure cost. PG&E and other parties recommend that Energy Division modify the requirement to include all disaggregated overhead costs in reported administrative costs in order to avoid this double-counting problem. The Final Report explains why this approach would be preferable to modifying the E3 calculator to allow the removal of overhead costs from tens of thousands of measure line items in the calculators.63
The workshop participants propose an action item on this issue that we find to be reasonable: The utilities should work on a joint request to the assigned ALJ (in R.06-04-010) and Energy Division to fix this problem by modifying the reporting requirements.
9.3. Direct Install Costs in the TRC Test
During the review of the utilities' June 1, 2005 portfolio plans, Energy Division's consultant (TecMarket Works) pointed out an anomaly for selected programs where the TRC was greater than the PAC. Given the definition of these tests (see above), the opposite should generally be true because the PAC test does not include the costs incurred by participating customers, while the TRC test does includes these costs. The exception to this general rule can happen given the SPM definition of the TRC test when very large "transfer payments" between non-participating and participating ratepayers occur. But as discussed below, this should not be a frequent occurrence if the proper definition of transfer payments is used and installation costs are accounted for appropriately.
TecMarket Works determined upon review that "the condition is E3-based and is associated with program conditions that occur when an incentive equals the full cost of the measure."64 TecMarket Works concluded that "this calculation approach appears to be different than the calculation approach described in the Standard Practice Manual" and that "there is a need to confirm with the [utilities] the calculation approach that should be used to assess the portfolios and make that approach consistent in the E3 calculator and in the Standard Practice Manual."65
This issue was discussed during the workshop process and addressed in DRA's written comments. Parties now appear to agree that this was not an error in the E3 calculator, but rather an issue with how costs are defined in direct installation-type programs and in particular, how those costs are defined when the sum of direct install costs plus rebates/incentives exceed the incremental measure cost.
In its written comments, DRA characterizes this anomaly as one arising from the SPM definition of the costs that comprise the TRC test. According to DRA, the TRC test "excludes as a cost ratepayer dollars paid to a program participant."66 Based on this understanding of the TRC test, DRA goes on to describe the following scenario for programs where participating customers incur no out-of-pocket expenditures:
"If a program implementer makes a lump sum incentive payment to contractors that covers all costs associated with a retrofit at no cost to the customer, that lump sum incentive payment will not be included as a cost into the TRC. Under such a scenario, the TRC would be greater than the PAC, because the TRC would exclude as a cost ratepayer dollars paid to a program participant and there are zero net participant costs, whereas the PAC would include ratepayer dollars paid to a program participant as a cost to the administrator. The resulting TRC net resource benefits would also exclude incentive payments as part of the program costs and therefore would be superficially high for such `no cost' retrofit programs."67
DRA urges the Commission to consider instituting a cap on participant incentive amounts. In DRA's view, such a cap would serve to discourage program implementers or utility program administrators from shifting program funding into "no cost" retrofit programs to increase TRC net resource benefits. DRA also recommends that the input fields for the E3 calculator be revised to separately capture the incremental equipment cost of the energy efficiency measure as well as the installation costs.
Based on the record in this proceeding, we find that the treatment of costs in the TRC test has caused some anomalies in E3 model calculations that can, and should, be corrected for future applications of the TRC test and the E3 calculator. However, we do not agree with DRA's framing of the problem as a definitional issue that arises from the SPM.
The SPM is very clear on what the TRC represents, as are our Rules. The TRC test of cost-effectiveness includes all costs associated with the energy efficiency activity, whether paid for out-of-pocket by program participants or by non-participants through the authorized revenue requirements that fund the programs. 68
The only costs that are excluded in the TRC test are those "incentives" that are to be are considered and treated as transfer payments. The SPM specifically directs that such incentives are restricted to include "only dollar benefits such as rebates or rate incentive (monthly bill credits)."69 The conceptual basis for ignoring transfer payments in the development of the TRC is similar to the basis for ignoring tax credits in the Societal version of the test. That is, when some taxpayers receive cash transfers (in the form of a tax credit) as a result of higher taxes paid by others, economic theory suggests that those transfers be excluded when calculating the costs and benefits of the investment from the societial perspective. Historically, the SPM has incorporated a similar concept with respect to cash rebates to participating customers in the TRC test. That is, they have been excluded on both the benefit and cost side of the TRC equation, and considered to be a transfer payment between participating and non-participating customers.
In order to more fully explore the anomalies observed in the E3 calculator results for TRC cost-effectiveness and discuss ways to correct them, as well as respond to some of the comments on the draft decision on this issue, we need to further illustrate with numerical examples what the TRC and PAC tests intend to capture in their respective formulas. So, in a very simplified example, if the resource benefits are $3,000, the participant's measure installation cost is $2,000, the program administration cost is $100 (not including the cash rebate) , and the participating customer receives a $1,000 cash rebate for installing the measure, the TRC equation before cancelling out the cash rebate as a transfer would look like this:
Benefit side: $1,000 + $3,000
(Benefit to participant of cash rebate + Resource benefits to all ratepayers)
Cost side: $2,000 + $100 +$1,000
(Participant's cost + Program admin cost (not including rebate) + Cost to non-participating customers of cash rebate)
By treating cash rebates as a dollar transfer payment, the SPM formula simply drops the $1,000 payment from both the benefit and cost side of the equation, producing TRC net resource benefits in this example of $900 ($3,000-$2,100) and a TRC benefit-cost ratio of 1.428 ($3,000/$2,100).
The PAC test, on the other hand, includes the cash rebate to the participating customer in calculating costs, but ignores the participant's costs. This is because the perspective of this test is the impact of the energy efficiency investment on utility revenue requirements. While the cash rebate to participating customers increases those requirements, the measure installation costs paid by the participant do not. The participant benefit of receiving a cash transfer payment from non-participating customers is not part of this test's perspective, so it never shows up on the benefit side of the equation at all.
Accordingly, for the simple numerical example presented above where the customer installs the measure and gets a cash rebate of $1,000, the PAC equation would look like this:
PAC Benefit side: $3,000
(Resource benefits to all ratepayers)
PAC Cost side: $100 +$1,000
(Program admin cost (not including rebate) + Cash rebate to participating customer)
Therefore, PAC net benefits would be $1,900 ($3,000 - $1,100) and the PAC benefit cost ratio would be 2.73 ($3,000/$1,100).
Prior to electric industry restructuring in the mid-1990s, most of the energy efficiency resource programs were similar in design to this numerical example-that is, participating customers would receive cash rebates to install energy efficient measures and equipment. Therefore, the term "incentive" and "rebate" were generally used interchangeably in the discussion of program costs and in the application of the SPM tests of cost-effectiveness. This is no longer the case, as pointed out in the workshop comments and discussion. Today, there are other forms of providing incentives to participating customers as well as other market actors purchasing and installing the equipment for the programs, resulting in misunderstandings and inconsistencies in how costs are being accounted for in the SPM tests and E3 calculator inputs. However, the manner in which the program is delivered or the rebate is provided to the customer should not result in different cost-effectiveness results, except in the very limited instances discussed below.
Let us look at the same simple numerical example under an early replacement "direct install" program design, where a third-party contractor replaces a customer's inefficient air conditioner with more efficient model. We assume that the resource benefits are $3,000, as in the prior example. We also assume that the utility incurs $100 in program administration costs. The utility authorizes the contractor to pay rebates of $1,000 on each installation. The contractor installs the unit at a cost of $2,000. The customer is presented with a bill for the $2,000 installation costs minus a $1,000 rebate. The contractor bills the utility for the $1,000 rebate given to the customer.
The SPM specifically states that "If the incentive is to offset a specific participant cost, as in a rebate-type incentive, the full customer cost (before the rebate) must be included in the PC1 [participant cost]."70 Consistent with the SPM formulas and definitions, the TRC and PAC tests would be calculated exactly the same as the example presented above for a program where the customer installs the equipment/measure instead of the third-party contractor, and receives a cash rebate:
TRC benefits: $3,000 PAC benefits: $3,000
TRC costs: $2,000 + $100 (Participant Costs + Program admin. )
PAC costs: $100 + $1,000 (Program admin. Costs + Cash rebate to participating customer paid through contractor)
TRC net benefits: $900; TRC benefit/cost ratio: 1.428
PAC net benefits: $1,900; PAC benefit/cost ratio: 2.73
Now let us look at an example where the direct install program does not bill or collect from the customer for any portion of the costs. Under both the TRC and PAC tests, the full $2,000 measure installation cost should appear as program administrator cost (rather than a participant cost), in addition to the $100 program administration costs. There would be no transfer payments or participant costs at all based on the SPM definition of these terms. The TRC test results would be the same as in the above examples. However, because the program results in higher utility revenue requirements (because now participants are incurring zero out-of-pocket costs), the PAC test results are not as favorable as in the previous two examples. In fact, the TRC and PAC test results would be identical to each other, as indicated below:
TRC benefits: $3,000 PAC benefits: $3,000
TRC costs: $2,000 + $100 (Direct install costs paid by utility + Program admin. costs)
PAC costs: $2,000 + $100 (Same as above)
TRC net benefits: $900; TRC benefit/cost ratio: 1.428
PAC net benefits: $900; PAC benefit/cost ratio: 1.428
These numerical examples serve to illustrate what should be obvious: A direct install program where the utility or its contractor performs the installation of a measure should not be more cost-effective from a TRC perspective than a rebate program that provides a cash rebate to the customer up to the full cost of installation. We recognize that there may be limited instances for program design purposes where the cash rebate to the customer exceeds the measure installation cost. Under these circumstances, the TRC results will be the same for both direct install and the rebate program (all other things being equal), given the transfer payment treatment of cash rebates in the SPM. However, the PAC test will favor the direct install program. It was precisely to address these types of circumstances that we adopted the "Dual Test" of cost-effectiveness in our policy rules. Those rules recognize that both the TRC and PAC tests of cost-effectiveness need to be considered when evaluating program proposals, in order to ensure that program administrators and implementers do not spend more on rebates/cash incentives than absolutely necessary to achieve TRC net benefits.71
The discussion above also points out that when the SPM definition of transfer payments is properly implemented in the TRC test, participant costs are expected to be "non-negative." We recognize that there may be isolated instances where the energy efficiency measure actually costs less than the standard efficiency equipment, as PG&E points out in its comments on the draft decision.72 However, one would not expect to see negative participant costs for the vast majority of measures or in the evaluation of program cost-effectiveness calculations where there is a mix of measures, if costs are inputted correctly into the E3 calculator and transfer payments are properly restricted per the SPM definition.
DRA's scenarios presume that if the participant pays no out-of-pocket costs under a direct-install program, then all of the costs associated with the equipment/measure installations simply disappear from the TRC cost-side of the equation. As discussed above in our third numerical example, that certainly should not be the case. Further, we note that this is not the case when the TRC test is performed for Low-Income Energy Efficiency programs, where participants generally incur no out-of-pocket expenditures for the installation of energy efficiency measures.
DRA also claims that when the customer rebate exceeds the equipment/measure installation costs, this creates "a distorted relationship between the TRC and the PAC benefit-cost ratios."73 This should also not be the case if the SPM cost components are inputted into the E3 calculator in a manner consistent with the definition of both tests. Again, the TRC test reflects all participant and non-participant costs, meaning that the full resource costs of the energy efficiency investment must show up somewhere in the TRC cost-side of the equation with the limited exception of transfers of dollar benefits (rebates/monthly bill credits) to participants.
In our view, these clarifications speak to the need to ensure that the program cost components and transfer payments are properly entered into the E3 calculator (or in other platforms for calculating and reporting cost-effectiveness results) consistent with the SPM formulas and definitions, rather than the need to cap incentive payments, as DRA proposes. As discussed in Section 10.2, we request that Joint Staff, the utilities and their program advisory/peer review group members explore ways in which this can be best accomplished through technical workshops. There may also be refinements to the E3 calculator that can serve to flag potential input errors and inconsistencies (e.g., negative participant costs, incongruous differences between TRC and PAC test results), that can assist in the quality control of input data. These refinements should be considered and presented during the E3 calculator updating process, discussed in Section 11 below.
We emphasize that today's discussion of the TRC and PAC tests of cost-effectiveness does not speak to the design of programs (or is intended to cap incentives in any manner). Instead, it speaks to need to ensure that all costs are inputted into the E3 calculator, or any other calculation platform for the SPM tests, in a manner that is consistent with the SPM formulas and definitions, as discussed above.
59 The latest version of the SPM (California Standard Practice Manual: Economic Analysis of Demand-Side Programs and Projects, October 2001) is posted on the Commission's website at: http://www.cpuc.ca.gov/static/energy/electric/energy+efficiency/rulemaking/eeevaluation.htm.
60 D.05-04-051, Attachment 3, Rule IV.2. As noted in footnote 7 of the Rules, the TRC test looks at the "incremental" measure cost (not the full cost) when an energy-efficient appliance or measure promoted through the program represents a replacement "on burn out" of the participant's existing appliance/measure. In other words, for these "replace on burn out" installations, the measure cost is the additional (incremental) cost of the equipment/measure relative to the standard (less efficient) appliance/measure that would have been installed, without the financial incentive or outreach program. Full measure/equipment costs are only used in instances where the program causes the participant(s) to do what they would not have done anyway (or at least not in the near future, e.g., 5 years), such as replace a working air conditioner with a more efficient one.
61 Ibid., Rule IV.3.
62 See Administrative Law Judge's Ruling on Reporting Requirements, issued February 21, 2006 in R.01-08-028, Appendix: Allowable Costs.
63 See Final Report, p. 34.
64 TecMarket Works Report, p. 34.
65 TecMarket Works Report, p. 14.
66 Comments of DRA in Response to the ALJ's Ruling Soliciting Preworkshop Comments on the Draft Report on the 2006 Update to Avoided Costs and E3 Calculator, March 9, 2006 (DRA Pre-Workshop Comments), p. 7. See also: Comments of DRA in Response to the ALJ's Ruling Soliciting Postworkshop Comments on the E3 Report on 2006 Update to Avoided Cost and E3 Calculator, March 27, 2006 (DRA Post-Workshop Comments), p. 9.
67 Id. See also: Comments of DRA in Response to the ALJ's Ruling Soliciting Postworkshop Comments on the E3 Report on 2006 Update to Avoided Cost and E3 Calculator, March 27, 2006 (DRA Post-Workshop Comments), p. 9.
68 SPM, p. 18.
69 SPM p. 11 (footnote 3 on page 11); 21.
70 SPM, page 11, footnote 3.
71 See D.05-04-051, Attachment 3, Section IV. In its comments on the draft decision, SCE correctly points out that a program may pass the TRC test but fail the PAC test under these circumstances, and therefore the draft decision proposed treatment of cash rebate costs in the TRC test was not fully consistent with the SPM. However, SCE's comments fail to acknowledge the more fundamental problem the draft decision identified; namely, the inconsistent treatment of incentives and participant costs in E3 calculator inputs and the calculation of TRC test results, particularly for direct install programs.
72 PG&E gives the example in DEER of double pane clear windows and direct evaporative coolers, tankless gas water heaters, among others. However, a closer examination of the DEER dataset reveals that the incremental measure cost is not negative (set at 0) even when the difference in equipment cost is negative. As noted in the SPM, the equipment cost is only one element or the measure or participant cost.
73 Comments of DRA in Response to the ALJ's Ruling Soliciting Postworkshop Comments on the E3 Report on 2006 Update to Avoided Cost and E3 Calculator, March 27, 2006, p. 9.
