ATTACHMENT 4
Review Protocol for
Impact and Market Effects Evaluation Studies
Joint Staff intends to use the following process after a contractor has been selected to conduct a specific set of evaluations for Impact and Market Effects studies. Steps 1-6 will be followed for all interim findings, steps 2-9 will be followed for all final reports.
1. For Each Study, Joint Staff Identifies Appropriate Experts Who Can Provide Input as Needed.
Joint Staff (JS) obtains input from program implementers, administrators and (as needed) other technical expertise in order to coordinate data collection and provide advice on analysis during the course of the study.
2. Contract Manager Reviews Draft Report
Once analysis is complete, the evaluation contractor sends a draft report to the JS contract manager for review and incorporates comments from the JS contract manager.
3. Study Team Conduct Verification Review on Revised Draft
After the JS contract manager approves the draft report, the evaluation contractor sends a verification draft to appropriate program implementers, administrators and other experts as directed by the JS contract manager to request corrections of factual errors or raise questions.
4. Revised Draft Report Posted and PAG Notification
The JS contract manager directs the evaluation contractor to make the necessary changes and post the revised draft report on the website. Program Advisory Group (PAG) members are notified of the report and given at least seven days to review the report before the results are presented in public.
5. Presentation of Revised Draft and Public Comments
The evaluation contractor presents the latest version of the draft report (in a teleconference or via email) to gather public comment from interested parties on the load impact or market effects study results.
6. Review of Public Comments and Final Report Released
JS contract manager reviews public comments and directs contractor to make any necessary changes stimulated by the comments. After incorporating the revisions that are requested by the JS contract manager, the evaluation contractor sends the final report (including a one-page summary of key recommendations and findings) to the contract manager, who will approve the final document for public release and web-posting.
7. Program Administrator Response to Final Report
Within 60 days of public release, program administrators will respond in writing to the final report findings and recommendations indicating what action, if any, will be taken as a result of study findings. JS can choose to extend the 60 day limit if the administrator presents a compelling case that more time is needed and the delay will not cause any problems in the updating schedule. In this follow-up response to each study, administrators should note any concerns they have over specific report findings and indicate whether they agree with the final load impact estimates for the programs in question.
8. Dispute Resolution
Disputes over the study results for determining savings goal achievements and/or performance basis will be addressed in the Annual Earnings Assessment Proceedings (AEAPs), or successor proceedings. Program Administrators shall file the response required under Section 7 above in the then-pending AEAP. The assigned Administrative Law Judge (ALJ) will determine the scope and schedule for resolving disputes through either formal or informal processes. To facilitate the dispute resolution process, Energy Division will contract with the appropriate technical expert(s) to provide the ALJ with EM&V technical assistance in the AEAP proceedings. This contract shall be paid with the EM&V funds authorized by the Commission.
9. All Relevant Documents Made Available
Energy Division ensures that the final report findings, recommendations and portfolio administrator responses are available at a central location or website and will provide summary reports that are needed for planning the next program cycle and as requested by the Commission.
(END OF ATTACHMENT 4)
