10. Other Issues
Two additional issues were discussed during the workshops and in comments, which we address below.
10.1. Appropriate Calculation Platform
The Final Report addresses the issue of what calculation platform to use for the ex ante evaluations and submissions of portfolio and program plans, for example, in preparation for the 2009-2011 program cycle. The consensus among workshop participants is that the benefits of the E3 calculator outweigh the shortcomings of a platform based on Excel spreadsheets, at least for the near term.74
We concur with this approach. As noted in the Final Report, some of the shortcomings can be addressed through a redesign of the calculator. In particular, the E3 calculator inputs and outputs can be separated from the calculation engine as a near term enhancement. In addition to the advantages noted in the Final Report, this approach would also facilitate the development of standardized "default" input values to improve quality control, as discussed further below. Therefore, we direct that this enhancement be made to the E3 calculator platform as part of the E3 calculator updating process described in Section 11.
Over the longer term, we may consider alternative platforms to use for the ex ante evaluations and submissions of portfolio and program plans. However, migration to another platform should not be decided until more information is known about the availability of new hourly load shapes, as well as the cost and effort needed for such an undertaking. In this phase of the proceeding we also intended to explore whether further refinements to the E3 calculator are needed to create a common planning/forecasting tool for use by utility portfolio managers, third-party implementers, regulatory staff and possibly program advisory and peer group members. This is clearly a longer term effort. Rather than initiate work on this effort today, we will focus on improving the E3 calculation platform currently in use, as discussed further below.
There was some debate during the workshops over whether the utilities should also be required to use the E3 calculator to generate the monthly, quarterly and annual reports required under the reporting requirements established for post-2005 energy efficiency activities. We will leave this issue to be resolved as suggested by workshop participants. The utilities should meet among themselves, E3 and Joint Staff on a common approach and tool for reporting that applies the SPM cost-effectiveness tests as described in this decision and can generate the required reporting information. The utilities and Joint Staff should jointly report back on the common approach and tool that will be used for this purpose by October 15, 2006. The report should be submitted to the assigned ALJ in R.06-04-010. The ALJ should consider this report in consultation with Joint Staff, and may take any additional steps necessary to ensure that a common approach and tool for reporting is implemented by the utilities in a timely manner.
10.2. Quality Control of E3 Calculator Inputs
The ex ante inputs to the E3 calculator for measure/equipment costs, measure savings, program administrator program costs, etc., are currently made by utility program managers (for utility-implemented programs) or by third-party implementers, subject to regulatory review when the portfolio and program plans are submitted. This involves the entering of hundreds of lines of data by numerous individuals. While there are guidelines for this data entry (e.g., the requirement in our Rules to use DEER values where available), the current process is less than ideal from a quality control perspective, as recognized by most of the workshop participants. The TecMarket Works and peer review group assessment of the June 1, 2005 filings served the Commission well in identifying some of the potential inconsistencies and errors in the E3 calculator inputs, but as workshop participants point out, there may be additional ways to assure greater quality control on an ongoing basis in the future.
One such approach may be to have the E3 calculator use a common or standardized data base to draw from as default value for most or all measures, and include the capability to "flag" the entries that differ from those values. This would still allow program managers and third-party implementers the flexibility to enter alternative values if they believe (and document) that the default values do not apply for the specific application or that better data is available. However, by flagging the data that does not utilize standardized default values, reviewers (Joint Staff, Energy Division consultants, peer review groups, etc.) can more easily identify areas for further examination. Currently, they must examine every single line of program input data and compare it to DEER data, for example, to evaluate whether or not the DEER values are used. The E3 calculator enhancement discussed in Section 10.1 (separating the input and output files from the calculation engine) would facilitate the development of such an approach.
Establishing a review process for the calculator inputs before they are entered into the calculator may also be a way to enhance the consistency of the data inputs, catch errors and address questions that arise about the input values. Based on DRA's comments, it appears that one area where such an "advance review" would be particularly useful is measure cost inputs. These and other approaches to quality control improvements should be explored collaboratively by Joint Staff, interested parties, the utilities and their program advisory/peer review groups in the coming months.
We direct the utilities to jointly plan and notice public workshops for this purpose during the third and forth quarters of 2006, and report back to the Assigned Commissioner and ALJ in R.06-04-010 on the consensus and non-consensus recommendations presented at those workshops no later than December 15, 2006. The workshop notice(s) should be sent to the service list in R.06-04-010 and to the utility program advisory group and peer review group members. The utilities are directed to jointly contract with appropriate technical expertise to assist in this effort so that options and specific implementation steps for quality control improvements can be fully explored during the workshops and in the report described below. In consultation with the utilities, Energy Division shall determine the specific EM&V budget category for funding this technical expertise. The Assigned Commissioner and ALJ in R.06-04-010 will consider the report in consultation with Joint Staff, and implement quality control improvements as they determine are appropriate and practicable.
74 These benefits and shortcomings are described in the Final Report, pp. 31-32.
