Summary of Results

The review of the 2006-2008 IOU portfolios reached the following conclusions.

1. The IOUs will meet the goals projected in their submitted portfolios if the following conditions apply:

a. If the Policy Manual's Net to Gross (NTG) ratios are confirmed to be substantially accurate via the ex-post evaluation efforts,

b. If the IOUs are able to wind up their programs to install the number of units projected consistent with the projected increase in program size,

c. If the IOU's measure-level energy savings projections are used as the basis for the accomplishment assessment (rather than evaluation confirmed measure-level savings), and

d. If the partnership, third-party, and bid programs achieve a rapid start-up and are able to meet their measure installation goals.

If these conditions do not apply and the evaluation-confirmed savings act to lower estimated savings by 20% or more, the goals may not be reached unless energy saving credits from the information, education, and marketing programs are applied.

2. The IOU portfolios are expected to be cost effective (TRC>1.0) even after evaluation-confirmed savings are applied.

3. The kW savings estimates across the IOUs are calculated using different dates for the period at which kW impacts are estimated. This condition causes the IOU kW impacts to be incomparable.

4. The majority of the electric and gas savings included in the statewide portfolio are non-DEER, IOU-calculated estimates. The portfolios are lacking complete measure estimate documentation, or the documentation provided does not provide a clear path for replicating the estimate for a significant number of these measures. This condition typically applies to measures that are difficult to estimate. However, all IOU submissions should contain a complete presentation of the calculation approach used for each non-DEER-based measure included in their portfolios. As a result, for some measures we are unsure if the impact projections are reasonable because the documentation was not clearly presented or was not presented early enough to allow a complete review. The CPUC should require an IOU-specific Energy Savings Dictionary with every filing, clearly presenting transparent and fully documented calculation formulas and input data and data sources so that CPUC staff can easily replicate and assess the calculations provided.

5. The KEMA potentials estimates do not reflect 2006 codes and standards conditions or current technology penetrations. These studies need to be updated if they are going to be used for 2006-2008 goal planning.

6. The energy savings estimates assume a growth economy consistent with the potentials study. If the economic growth projected is not realized, goal attainment is additionally at risk.

7. PG&E and SCE have moved to a Flagship approach to providing programs and have structured their portfolios around market sectors instead of individual programs operated separately. This is a positive change that will require expert management and new tracking system designs. The tracking systems will need to be expertly managed and populated if the CPUC's impact evaluation efforts are to succeed. The new tracking systems should be reviewed and approved by CPUC evaluation staff early in the program cycle.

8. Administrative costs seemed to be calculated differently by the IOUs and may not be comparable.

9. The CPUC should consider establishing a formal mid-course program design and funding change policy and convey that policy to the IOUs to be incorporated into their portfolios. That policy should allow the CPUC to function in their public oversight responsibilities, but provide a level of flexibility that allows the programs to be modified within a public approval process. The current portfolio planning process allows the IOUs, working with the Program Advisory Groups, to recommend the oversight function and process.

10. The current portfolios assume that the Policy Manual's NTG ratios accurately project savings potential. For several measures, the Policy Manual NTG values are not consistent with evaluation findings, causing projected savings to be higher than what the EM&V efforts will confirm. The NTG values need to be updated if they are to accurately predict program impacts. Moving to a single NTG value of 0.8 for all measures does not solve the issue.

11. There needs to an agreement and consistency of the calculation approach used in the E3 Calculators and the California Standard Practice Manual. At this time they do not appear to be consistent. Using the Policy Manual as a guide, we would not expect to see the PAC as a larger number than the TRC. Upon review of this issue, it appears that the condition is E3-based and is associated with program conditions that occur when an incentive equals the full cost of the measure, such as when a refrigerator is given away at no cost to the participant or when a program is offering incentives above the incremental cost of the measure

12. The IOUs do not have consistent energy impact estimates or estimate approaches at the measure level, even after adjusting for climate differences.

13. Some program TRCs are higher than we expected. However, the wide range of TRC results across the portfolio were expected as a result of the wide range of programs offered.

14. The IOU portfolios seem to be consistent with the incentive and reward structures in that they focus on kWh savings. However, the primary needs of the state may be kW impacts. To obtain high kW impacts, the incentive structure may need to focus on kW impacts. Emphasizing lowest cost kWh and high TRC values acts to focus the portfolio on fewer high performing measures, such as lighting upgrades. However, the portfolio appears to be reasonably well-balanced, considering the planning and award approaches in operation.

15. There is no contingency fund to allow for rapid capturing of short-term market opportunities without reducing funding for other planned efforts. The CPUC should consider establishing a contingency fund to allow programs to rapidly capture market short-term emerging market opportunities without needing to pull funds from planned activities or other programs. This fund should be capable of deploying resources in a few days when a new or emerging cost-effective market opportunity is identified.

16. Several measures incentivized in the IOU portfolios are already required by law (codes and standards). Whether these measures should be included in the portfolios is questionable. Because several of the measures in the portfolios are already covered by the new codes and standards, we are assuming that these are being incentivized because of lack of code compliance. If this is the case, it may be more cost-effective to operate compliance programs out of a governmental agency such as the CEC or through local jurisdictions. The CPUC should consider assessing the level of compliance in late 2005 or early in 2006 to determine if a compliance initiative should be launched.

17. This portfolio contains a significant increase in past program funding levels. We are concerned that program delivery ramp up may be slower than expected, causing 2006 savings to be difficult to deliver. If the IOUs are aggressive and manage the ramp up well, this may not be an issue. Previous program cycles have experienced start-up delays and logistical issues, slowing program implementation. We have concerns if these programs, as a whole, can acquire the projected savings for 2006.

18. There are very large budgets for marketing and outreach programs that have not yet demonstrated their value in producing direct or indirect energy effects. Before the CPUC approves a three-year budget for these programs, these efforts need to have an effects evaluation to document how energy consumption is directly or indirectly influenced by these efforts.

19. The On-Bill Financing program may pose a public image risk in that the ratepayers will be paying the IOUs over 8 percent return on their loan investment, when the risks of default are low and the consequences high.

20. There may be substantial savings opportunities by placing all programs that deal with construction or construction-related services into a coordinated statewide umbrella structure that delivers these efforts into the market.

21. There appears to be lost additional opportunities in some key areas. These include:

a. Agriculture programs,

b. Manufactured housing retrofits,

c. Manufactured housing, and

a. HID replacements.

We hope that the suggestions and issues identified in this report will further strengthen the portfolios and their ability to achieve their savings goals.

The remaining sections of this report provide more detailed information on the assessment process and findings.