| Word Document PDF Document |
ATTACHMENT 2
Performance Basis Protocol
For Verifying Performance Basis Parameters
And Joint Staff's Reporting Schedule
This protocol identifies when Joint Staff plans to verify various parameters that are used to calculate the performance basis for each portfolio administrator for the planning cycle 2006-2008. Joint Staff plans to provide two types of reports to verify the level of energy and peak savings achieved by programs and the performance basis for each administrator's portfolio of programs:
A. Verification reports - Three annual verification reports will serve to verify the number of measure installations and portfolio and program costs from the previous program year in August of 2007, 2008, and 2009.
B. Interim and Final Performance Basis Reports - These reports will provide Joint Staff's interim and final estimates of the net performance basis achieved for two snapshots in time: the first 18 months of the program cycle in the interim report and the full 36 months of the cycle in the final performance basis report. These reports will also provide information on the annual and cumulative levels of energy and peak savings achieved for this same time period.
The interim performance basis report will be published in March of 2008. Due to timing constraints, the interim report will not have sufficient data to confirm or verify all of the ex ante estimates of energy savings, load shapes and savings. In some cases, this will mean that ex ante estimates made at the time of program authorization will be used in the calculation of the interim performance basis. However evaluation consultants will be asked to develop evaluation plans that will update key parameters identified as uncertain in the planning process within this interim document. Thus, the interim document could contain updated parameter estimates based on 18 months of data collection for some or all of the following parameters:
1. Measure Installations
2. Program Costs
3. Unit Energy Savings/Measure Installation by Strategy
4. Program Level Estimates of Gross Energy Savings (product of 1 and 3)
5. Net-to-Gross Ratios by Program Strategy and/or Measure
6. Program Level Estimates of Net Energy Savings (produce of 4 and 5)
7. Load Factors or Daily Load Shapes used to transform annual savings estimates into peak savings estimates
8. Incremental Measure Costs
Evaluation contractors will not be asked to develop updated estimates of Avoided Costs or the Expected Useful Lives of Measures for use in the performance basis calculation. These values will be taken from the ex ante filings for useful life of measures and from the 2006 update of avoided cost values, per the Commission's direction.1
The final performance basis report will contain updated estimates for all of the seven parameters listed above for the 2006-2008 cycle. This report will be published on March 1, 2010. Consistent with the interim report, the final report will use ex ante values for avoided costs and expected useful lives of measures in the calculation of final performance basis for the administrator. Joint Staff will present updated estimates of performance basis, using a mix of verified and ex ante parameters, in each of the reports listed below.
The parameters to be verified in each of these reports are summarized in the following table. A more detailed description of how each parameter will be verified is presented after the table.
Evaluation Results Joint Staff Reporting Schedule
Report Date |
August 2007 |
March 2008 |
August 2008/9 |
March 2010 |
Report Title |
2006 Verification Report |
Interim Performance Basis Report |
2007/8 Verification Report |
Final Verification and Performance Basis Report |
Parameter |
Report Scope | |||
Verification of Measure Installations and Services Rendered |
Jan - Dec 2006 |
NA |
Jan - Dec 2007/8 |
PY 2006 - PY 2008 |
Program Costs |
Jan - Dec 2006 |
NA |
Jan - Dec 2007/8 |
PY 2006 - PY 2008 |
Measure or Unit Energy Savings and Peak Demand Reductions |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Program/Portfolio Energy Savings and Peak Demand Reductions |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Load Factors/Daily Load Shapes |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Incremental Measure Costs |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Avoided Costs |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Expected Useful Lives/Technical Degradation Factors |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Net-to-Gross Ratios |
NA |
Jan 2006 - July 2007 |
NA |
PY 2006 - PY 2008 |
Discussion of How Each Performance Parameter will be updated
1. Measure Installations - Program Administrators are expected to report on the number of measure installations and associated program costs throughout the 3-year program cycle. Joint Staff plans to have its contractors verify this information on measure installations by performing quality control checks on the measure installation inputs to the data base and verifying actual installations in a sample of customer premises using contact information provided by utilities. We expect Joint Staff verification efforts to lag the measure installation by 1 to 12 months, depending upon the type of project.
We expect that administrators will submit their reports to Energy Division or its EM&V contractors that include cumulative measure installations from the previous program year (2006, 2007, and 2008) on February 28th of each year.2 Joint Staff would plan to make its best effort to verify the installation counts by program and provide this interim estimate to each utility administrator on July 1st of each year and then publish the final estimate as part of its August report. Joint Staff would work with the administrators to resolve any misunderstandings or communication issues that might have led to differences in verified installations before developing an interim estimate of the performance basis for the portfolio in the August 1st report.
2. Program Costs
On an annual basis, Joint Staff plans to verify program cost estimates reported by each program administrator and will include non-confidential findings as part of its August 1st verification report.
3. Unit Energy Savings/Savings by Program Strategy - Utility program administrators have already provided estimates of the unit energy savings by measure or end-use and then used these estimates combined with forecasts of measure installations to develop program level savings estimates. Joint Staff plans to provide interim measure savings results in the first interim performance basis report in March 2008 and to provide final verification of the measure unit energy savings estimates for the entire program cycle in the final performance basis report in March 2010.
4. Program Level Estimates of Gross and Net Energy Savings
Joint Staff plans to conduct evaluations of the gross and net savings for each program in the utility portfolio. To the extent practicable, those findings will be broken out by program and/or program strategy. Interim results will be presented in the interim performance report in March 2008 and final results in March 2010.
5. Load Factors or Daily Load Shapes to Transform Annual Energy Savings Estimates Into Peak Savings Estimates
Joint Staff plans to estimate the peak load impacts from a variety of programs using the Gross Demand Savings Protocols. These protocols allow the evaluators to use secondary load shape data or primary interval meter data to estimate peak savings depending on the level of rigor selected by the evaluation team. Joint Staff will make interim results from these studies on an informal basis and then finalize the estimates in the performance basis reports. These peak savings estimates will be available at the same time as the estimates of program energy savings are published. In addition, measure or end-use level savings estimates may also be produced and reported in the interim or final performance basis reports.
6. Incremental Measure Costs
Joint Staff plans to verify the utility reported estimates of incremental measure cost on a spot check or sample basis to ensure consistency with the DEER estimates. In addition, Joint Staff plans to review and verify estimates of incremental cost for large industrial and commercial energy efficiency projects where ex ante estimates of incremental costs were not available.
7. Avoided Costs
Joint Staff will have its contractors verify that utility performance basis calculations utilize the adopted avoided cost time series (per the 2006 Update) whenever administrators are asked to provide an estimate of the performance basis of their portfolio.
8. Expected Useful Lives of Measures
Joint Staff plans to hire contractors to estimate survival functions for a selected set of measures using guidance from the expected useful live protocol. The goal is to estimate survival functions and ultimately useful lives for those measures that are forecast to be responsible for a significant proportion of the portfolio savings but were not covered by the most recent evaluation of useful lives completed in the last three years. These estimates will be used to update the ex ante estimates of useful life for the next program planning cycle but not to update the useful life estimates used in the 2006-2008 program estimates.
9. Net-to-Gross Ratio
Joint Staff plans to estimate net-to-gross ratios for each of the program delivery strategies as part of its load impact evaluations for each of the major program strategy groupings. In some cases, the net-to-gross ratios will also be reported for specific measures and or end-uses associated with a given delivery strategy, as appropriate. For example, the net-to-gross ratio for a downstream rebate program focused on increasing the sales of compact fluorescent lamps, might be available for a given program year, say 2006, but would need to be updated at the end of the program cycle to account for any changes in program delivery strategies in 2007 or 2008. The availability of these net-to-gross estimates is closely linked to the schedule for releasing estimates of gross and net program energy savings in the interim and final performance basis reports. These net-to-gross ratios will be combined with estimates of gross energy savings to yield net program savings estimates in the interim and final performance basis reports.
(END OF ATTACHMENT 2)
1 Per D.05-09-043, the program administrators are required to use the ex ante values for expected useful lives that were posted to the Commission's Database For Energy Efficiency website in July and August, 2005. (See p. 101 of that decision.) See also Section 8.8 of D.05-09-043 for a discussion of the avoided cost/E3 calculator refinements that will be undertaken in the avoided cost rulemaking (R.04-04-025) to update the ex ante forecasts of avoided cost for the 2006-2008 program cycle.
2 The frequency of reports on measure installations (e.g., monthly/quarterly) and the data transfer process (what data is submitted by program administrators directly to Energy Division, what data is sent directly to the EM&V contractors, etc.) will be established by the Reporting Requirements.