ATTACHMENT 3

Public Process Protocol for Impact Evaluation Study Scoping

Description of the Planning Process to Develop Specific Evaluation Studies and Budgets

For Impact Studies

1. Risk Analysis
Joint Staff and its consultants will conduct a risk analysis to identify key uncertainties in program strategy and measure savings estimates.

2. Peer Review and Expert Input on Risk Analysis
and Joint Staff Priority Assessment
After conducting an initial risk analysis with technical consultants, Joint Staff will schedule private meetings to gather input on the uncertainty and risks associated with the ex ante estimates of program savings included in the portfolio administrator filings. The purpose of these meetings is to help Joint Staff allocate evaluation resources to those programs, program strategies, measures, and parameters with the highest level of uncertainty and/or that constitute a significant portion of the total portfolio savings. Joint Staff will invite technical evaluation experts to provide their assessment on the level of certainty in current estimates of baseline unit energy consumption's (UEC's) measure energy savings estimates, measure or customer level participation estimates, net-to-gross adjustments, and expected useful life estimates for a select group of measures and program strategies. Joint Staff will then conduct a priority assessment and budgeting exercise that utilizes the results of the risk analysis.

3. Public Workshop on Risk Analysis and Priority Assessment
Joint Staff will hold a public workshop to solicit input on the risk analysis and the proposed evaluation priorities. Joint Staff will send out written materials that will familiarize parties with the process that was undertaken and the results that Joint Staff generated through that process in advance of the workshop. Joint Staff will refine or adjust their risk analysis and priority assessment taking into account feedback that parties provide during the workshop (and if possible in written comments.)

4. Study Scoping
Joint Staff will develop draft study scopes that reflect Joint Staff's assessment of uncertainties underlying the savings estimates. The study scopes will also identify the most important parameters to be verified in order to ensure that the program administrator savings can be assessed. This material will be posted on the Commission's website.

5. Public Workshop on Study Scopes
Joint Staff will hold a public workshop to solicit input on the study scopes. The study scopes will include at a minimum the matching of applicable protocols to programs, applicable level(s) of rigor, risk factors and corresponding budgets.¹

6. Final Evaluation Plans
After the workshop, Joint Staff will prepare and post FINAL impact evaluation plans to the CPUC website. These plans will then be used in the development of requests for proposals that Joint Staff or Energy Division will issue in order to hire consultants who will manage EM&V studies.

Joint Staff reserves the right to make changes based on comments or decide to remain with its original plans.

A proposed schedule for the above process is outlined on the following page of this document.

Risk Analysis/Priority Assessment/Study Scoping and RFP Schedule

Activity	Start Date*	End Date*
Joint Staff conducts risk analysis with consultants and holds meetings with ad-hoc technical review committees to refine risk analysis and identify key uncertainties in program strategy and measure savings estimates.	2/06
Joint Staff holds workshop and solicits comments on results of risk analysis in the form of prioritized list of program, program elements, program strategies, end-uses, measures, and measure savings parameters for EM&V.	3/06
Joint Staff prepares high level evaluation study plans based on results of risk analysis and considering parties' input. High level plans will list planned studies and high level scope of work.	5/06
Joint Staff holds workshop to present high level evaluations study plans and detailed revised budgets and to gather comments from parties.	6/06
Joint Staff prepares individual evaluation study plans to be put out to bid in next six months considering parties' inputs. Individual study plans will be detailed scope of work to be included in the RFPs.	6/06	Throughout evaluation cycle as necessary
Energy Efficiency Programs Launched	1/06	12/08
Develop and release evaluation RFPs based on detailed study plans and considering parties' inputs. RFP release will be staggered depending on the type of study.	7/06	Throughout evaluation cycle as needed
Enter EM&V contracts and review of contractor detailed evaluation plans.	8/06	Throughout evaluation cycle as needed
Program administrators report periodically to Joint Staff on program status, costs, program participations, etcetera.	Frequency to be determined by reporting requirements
Joint Staff EM&V expenditure and study description report	Quarterly through 12/10

*Note: Start and End Dates for this process are preliminary estimations only and may be adjusted.

(END OF ATTACHMENT 3)

¹ As discussed in the December 13, 2005 public workshop, these budgets are preliminary and will likely be set relatively high, based on estimations, with the expectation that these budgets will be adjusted in the future to reflect experience.