8. Reporting, Measurement, and Evaluation
8.1 Reporting
The utilities currently provide monthly reports on both their interruptible and price-triggered programs to Energy Division and the service lists for
R.02-06-001 and R.00-10-002. Because the reports are served on two different service lists and SCE provides two different reports, there is confusion about what reports to rely on. The reports reflect monthly expenses, accounts enrolled, MW forecasts, and details concerning the events called, if any. Because the monthly reports reflect the `upper bound' in terms of potential demand response, the MWs in the monthly reports are inaccurate, and at worst, misleading for planning and forecast purposes. For example, Demand Bidding Program MWs included in the reports assume every participant will bid when called, and will bid 15% of their average on-peak demand. Based on the program evaluation information to date, this result is unlikely. To be useful for planning purposes, the reports need to be modified to reflect a more realistic MW value for all the programs going forward.
Therefore, we direct the utilities to meet with CEC and Energy Division staff to determine alternative ways of reporting MWs and any other data that misstates load reduction capability or may be inaccurate. SCE should also only prepare one report (like the other two utilities). We recognize that while demand response programs are still fairly new and relatively untested, there is a tension between efforts to reach the day-ahead notification program MW goals, and realistic MWs for planning purposes. For now, we direct the utilities to, at a minimum, include both figures, demand response potential and expected/actual demand reduction when called, in their reports. This will allow us to better assess how to reflect these programs in future long term procurement plans.
8.2 Measurement and Evaluation
The utilities propose a joint utility measurement and evaluation (M&E) program similar to what is currently in place, with a focus on assessing marketing and implementation, surveys of participants and non-participants, and determining load impacts of the programs. The utilities propose evaluating the following programs: CPP, Demand Bidding Program, CPA DRP, Technical Assistance and Incentives, IDSM and the FYPN Campaign. SDG&E proposes M&E for its interruptible programs as well. The utilities also propose an annual summary report (due the 1st quarter of the following year).
Although SDG&E was the only utility to propose evaluation of its interruptible program for load impacts, the time appears ripe to do a more comprehensive analysis of reliability-triggered programs in 2005 so that we can readily compare the costs and benefits between programs. In comments on the draft decision PG&E supports a review of reliability triggered programs. The evaluation of the programs will only be useful to the extent that the programs are triggered. Given the relatively few times the programs were triggered in 2004, the evaluation findings and conclusions have several caveats and disclaimers, making the M&E effort somewhat limited in its usefulness. As a result, we will authorize funding for process and impact evaluation of both reliability and
day-ahead triggered programs but direct the utilities to work with Energy Division and CEC staff to identify which specific programs would provide the most useful analytical information based on how frequently the programs are triggered. To the extent possible we also encourage coordination with the Commission's avoided cost proceeding (R.04-04-025). We approve the general scope of proposed M&E activities.
