MEG/jva 1/27/2005

BEFORE THE PUBLIC UTILITIES COMMISSION OF THE STATE OF CALIFORNIA

Order Instituting Rulemaking to Examine the Commission's Future Energy Efficiency Policies, Administration, and Programs.

Rulemaking 01-08-028 (Filed August 23, 2001)

ADMINISTRATIVE LAW JUDGE'S RULING ISSUING COMPILATION
OF E-TABLE DATA FOR PRE-1998 ENERGY EFFICIENCY
PROGRAMS AND REQUESTING FURTHER COMMENT

During 2004, Energy Division conducted several workshops to address evaluation, measurement and verification (EM&V) protocol issues in this proceeding. From my reading of the workshop reports and written comments, there is agreement that certain performance parameters, such as program costs and number/type of measure installations, should be adjusted or "trued-up" based on verified costs and installations on an ex post (post-installation) basis. This true-up would occur before the performance of a program, program administrator or implementer for a particular program cycle is assessed, or any potential incentive or earnings associated with that performance is awarded.

The record also indicates general agreement that ex post measurement studies of per unit kilowatt (kW) and kilowatt-hour (kWh) savings assumptions should be undertaken to inform future program planning and resource planning assumptions. Notwithstanding this general agreement in principle, I recognize that there are differences among parties as to what types of methodologies these studies would entail, the frequency of performing such studies, and whether exceptions should be made to allow for the use of ex ante savings assumptions (i.e., forecasted prior to measure installation) in some instances where, for example, the per-unit savings have been well established.

Nonetheless, it appears that the threshold issue regarding ex ante versus ex post measurement of performance basis revolves around the following: Should the results of ex post studies of per unit kW and kWh savings be used to adjust ("true-up") the performance basis of a prior program year? In considering this issue, I believe that the record in this proceeding would greatly benefit from an analysis of the pre-1998 program data that has been submitted in the E-Tables appended to Annual Earnings Assessment Proceeding applications in accordance with our reporting requirements.

These E-Tables provide information in a standardized format, for each program year and by utility, as follows:

(1.) Ex ante performance basis (or "performance earnings basis," "PEB"), based on forecasts of all performance parameters for the program year in question. These are the forecasts during the program planning process when programs are selected for funding. The PEB under the pre-1998 protocols represents a net benefits calculation based on a weighted average of two tests of cost-effectiveness. Simply put, the PEB is calculated in terms of the dollar savings (resource benefits minus costs) realized by ratepayers from the programs;

(2.) PEB adjusted for ex post verification of program costs and program participation (including types and numbers of measures installed at each location), but still using the ex ante forecasts of lifecycle kW and kWh savings per measure (or "per unit") presented in (1) above;

(3.) PEB adjusted for verified costs, verified program participation and the results of ex post first-year load impact studies; and

(4.) PEB adjusted for all the performance factors in (3) plus the results of ex post persistence studies. The combination of the first-year load impact studies and subsequent persistence studies produce the ex post estimates of lifecycle kW and kWh savings that are applied to the installed energy efficiency measures.

At my request, utility staff has compiled the data from the E-Tables for the pre-1998 program years and presented it in a summarized format, along with E-table references. (See Attachment to this ruling.) Over the next few days, San Diego Gas & Electric Company (SDG&E) will also post all the underlying E-Table data by utility and program year at: www.sdge.com/regulatory.

By this ruling I am soliciting comment by interested parties on this supplemental information and, in particular, on whether this information reinforces, modifies or otherwise impacts your views on the threshold issue discussed above. Comments should also discuss whether you agree with my general observations and conclusions from the data, as described below-and if not, why not.

My preliminary review of the data indicates that the largest true-up adjustments to the ex ante performance basis occurred in the first earnings claim, where actual program costs and verified program participation were substituted for the ex ante values. For example, in 1996, the ex ante ("target") PEB for Pacific Gas and Electric Company (PG&E), Southern California Edison Company (SCE), SDG&E and Southern California Gas Company (SoCalGas) combined was a forecasted $140,078,000 in net benefits. Adjustments based on verified costs and participation (types and number of measures actually installed) increased the ex ante estimate by 113% to $298,944,000 which accounted for 96% of the ex post net benefit value ($311,540,000) for that program year. (I will refer to PG&E, SDG&E, SCE and SoCalGas collectively as the "investor-owned utilities", or "IOUs" throughout this ruling.)

The data also indicates that, for the IOUs combined, the results of the first-year load impact studies (conducted for the second earnings claim) and the persistence studies (conducted in the third or fourth year) generally cancelled each other out over time. That is, while the ex ante assumptions of first-year load impacts were higher than the subsequent ex post load impact studies revealed, the ex ante assumptions of expected useful life, measure retention and technical degradation were lower than the corresponding ex post values produced by the third or fourth year persistence studies. By 1996 and 1997, these forecasting errors cancelled each other out, producing ex post values for kW and kWh lifecycle savings quite close to the ex ante assumptions used for the programs.

For example, in 1996, the first earnings claim produced a performance basis of $298,944,000 in net benefits using ex ante per unit savings assumptions. The first-year load impact studies performed for the second earnings claim reduced this estimate by 9% and the third-year persistence studies raised it up again by 15%, for an ex post estimate of $311,540,000 in net benefits. This represents a forecasting error of +4%, meaning that the ex ante estimates of kW and kWh per unit savings for that program year were 4% lower than the corresponding ex post values on an IOU-combined basis. For 1997, the first earnings claim produced a performance basis of $258,981,000 using ex ante per unit savings assumptions. The first-year load impact study performed for the second earnings claim reduced that estimate by 19%, and the third-year persistence study raised it up again by 14%, for an ex post value of $240,081,000 in net benefits. This represents a forecasting error on the order of -6.4%, meaning that the ex ante estimates of kW and kWh per unit savings for that program year were 6.4% higher than the ex post values produced by subsequent studies.

In sum, the available data indicates to me that, for the IOUs combined, the ex post reevaluation of lifecycle kW and kWh savings conducted for the pre-1998 programs did not produce significant adjustments to ex ante forecasts of net resource benefits once the actual program costs and program participation had been verified. This is not to imply that reliance on ex ante kW and kWh savings assumption is without some inaccuracies. Had the Commission relied on this approach (while truing up cost and participation parameters) for the 1994-1996 program years, we would have underestimated program net benefits and associated earnings for the IOUs combined, and slightly overestimated the net benefits and earnings for program year 1997.¹

One can see this by comparing the PEB for the first-earnings claim relative to the PEB calculated after the load impact and persistence studies were performed in the third or fourth year after program implementation. As indicated in the Attachment, the net benefits for program year 1994 calculated after adjusting cost and participation parameters (first earnings claims) are $497,017,000. After further adjusting net benefits based on load impact and persistence studies, the net benefits for that year are $600,602,000. Hence, the net benefits calculated with ex ante per unit kW and kWh savings estimates captured only 83% of ex post net benefits associated with 1994 programs, for the IOUs combined. For program years 1995 and 1996 this percentage was 87% and 96%, respectively, also representing an underestimation of savings for those years. In 1997, this percentage was 108%, indicating that the ex ante estimates of kW and kWh savings used in that year slightly overestimated savings for that one program year.

As discussed above, I am interested in receiving supplemental comments on the E-table data compiled in the Attachment, on my preliminary observations about this data, and on any impact this additional information has on your position regarding the issue: Should ex post reevaluation of per unit kWh and kW savings be required in assessing the performance basis for prior program years?

IT IS RULED that:

1. Interested parties may file supplemental opening and reply comments on the EM&V issues described in this ruling and the attached compilation of E-Table data. Opening supplemental comments are due no later than February 18, 2005 and reply supplemental comments are due no later than February 25, 2005.

2. The Electronic Service Protocols attached to the Assigned Commissioner's Ruling dated December 22, 2003, which can be viewed at the Commission's website ( www.cpuc.ca.gov) apply to the documents required by this ruling.

Dated January 27, 2005, at San Francisco, California.

		/s/ MEG GOTTSTEIN BY LYNN T. CAREW
		Meg Gottstein Administrative Law Judge

CERTIFICATE OF SERVICE

I certify that I have this day served the attached Administrative Law Judge's Ruling Issuing Compilation of E-Table Data For Pre-1998 Energy Efficiency Programs and Requesting Further Comment on all parties of record in this proceeding or their attorneys of record by electronic mail to those who provided electronic mail addresses, and by U.S. mail to those who did not provide e-mail addresses.

Dated January 27, 2005, at San Francisco, California.

/s/ JANET V. ALVIAR

Janet V. Alviar

NOTICE

Parties should notify the Process Office, Public Utilities Commission, 505 Van Ness Avenue, Room 2000, San Francisco, CA 94102, of any change of address to ensure that they continue to receive documents. You must indicate the proceeding number on the service list on which your name appears.

¹ The utility-specific numbers in the Attachment reveal that most of the underestimation was attributed to PG&E's ex ante assumptions of kW and kWh savings (relative to the results of subsequent ex post studies) which-given the relative size of PG&E's programs--more than offset the overestimations of kW and kWh savings estimates associated with SoCalGas and SCE's ex ante assumptions.