Assignment of Proceeding

Dian M. Grueneich is the assigned Commissioner and David M. Gamson is the assigned Administrative Law Judge in this proceeding.

Findings of Fact

1. EM&V is an important part of the Commission's energy efficiency program for several reasons. First, it is necessary to determine whether and how well current individual programs are working, both in terms of saving energy and in comparison to projections. Second, EM&V is critical in considering how to improve programs and for development of new measures. Third, EM&V is used on a broad level to measure whether the IOUs are meeting, on a portfolio basis, the overall energy savings goals established by the Commission. Fourth, EM&V results are used to determine whether IOUs should receive rewards or pay penalties as part of the energy efficiency shareholder incentives plan developed by the Commission.

2. D.05-01-055 and D.05-11-011 set forth the rules for EM&V for the 2006 through 2008 energy efficiency program.

3. D.09-09-047 adopted energy efficiency portfolios for 2010 through 2012 for SCE, SoCalGas, SDG&E, and PG&E. That decision established the Commission's goals and core objectives for EM&V for the 2010 through 2012 portfolios.

4. On July 7, 2009, an ALJ Ruling sought comment on an ED "straw proposal" on EM&V issues. In the July 7 Ruling, parties in this proceeding were given notice that the Commission may in this proceeding adopt changes that would modify D.05-01-055. To that end, the July 7 Ruling with its attachments was served on the service list in R.01-08-028 (the proceeding in which D.05-01-055 was issued).

5. A budget of $125 million, or 4% of the overall portfolio budgets, for 2010 through 2012 EM&V was adopted in D.09-09-047, subject to review in this decision.

6. In previous energy efficiency cycles, each utility's EM&V budget was determined by its proportional share of the total EM&V budget approved by the Commission, with the proportion equal to its proportion of total program budgets. For the 2010 through 2012 energy efficiency cycle, these proportional amounts are: 43% for PG&E; 39% for SCE; and 9% each for SDG&E and SoCalGas.

7. ED and the IOUs prepared an EM&V plan which was jointly submitted to the assigned ALJ and issued for comment via Ruling. The Joint Plan is responsive to the Commission's stated desire in D.09-09-047 to make near-term improvements in order to streamline EM&V processes, and enhance timeliness, transparency and consistency across EM&V work products and to take a fresh look at several aspects of our EM&V activity in California for the upcoming program cycle, to reduce unnecessary burden on staff and other resources, and streamline our EM&V processes.

8. Both ED and IOUs should have specific and defined roles in EM&V.

9. IOUs continue to have a vested interest in the outcome of EM&V studies, as these studies are used to determine the level of energy efficiency shareholder incentives.

10. As program administrators and implementers, the IOUs have the data and the expertise to provide critical input to EM&V studies. IOUs also have a role, along with other energy efficiency constituents, to use the data gathered in EM&V efforts to improve program development and implementation.

11. There is evidence of overlap between IOU and ED EM&V activities that reduces the effective use of ratepayer funds.

12. EM&V studies should help form the basis for improvement of energy efficiency programs by showing what works well and what does not. It is important that this information be made available to stakeholders to the greatest degree feasible without compromising confidential information.

13. The IOUs require 15% (or $18.75 million) of the $125 million EM&V budget for 2010 through 2012 to maintain staffing levels.

14. $34.3 million is a reasonable estimate for the funding needed for IOUs to perform EM&V studies in 2010 through 2012, including the amount needed to maintain staffing levels.

15. IOUs have in the past conducted some or all of the program design and market assessment studies and early EM&V activities.

16. The determination of energy savings involves a variety of technical assumptions and calculations, with a high potential for differing opinions.

17. Currently, disputes between the IOUs and ED regarding EM&V studies are resolved by ED, with no specific process for appeal. All parties agree that increasing contentiousness in this area requires the Commission to create a new EM&V dispute resolution process.

18. ED is not a party to Commission proceedings. If an EM&V dispute is made formal through a Motion or other action by a party, ED would not have a right to file comments disputing the party's version of the dispute.

19. ED provided recommendations regarding IOU involvement in ED EM&V studies in Section C of the ED Straw Proposal "Stakeholder Input Process and Approval of EM&V Projects," which are complemented by the informal interactions proposed in the Joint Plan.

20. ED provided recommendations regarding determining the level of public vetting for EM&V projects.

21. ED recommends that the Commission require that all EM&V-related projects, regardless of funding source, adhere to the same policies and procedures as EM&V funded projects.

22. ED recommends that it review past projects with the IOUs to determine the energy efficiency incentive threshold above which customer participation in evaluations would be obligatory.

23. D.05-01-055 established a policy that prohibited firms engaged in energy efficiency work from performing both program impact evaluations and program implementation. This policy at times limits the ability to recruit program implementers to collect data needed for EM&V.

24. Avoided costs adopted in D.06-06-063 need to be updated for the purpose of calculating benefits from the 2009 bridge funding period, including an updated GHG adder, and for 2010 and forward.

25. There are significant intersecting issues with the IOUs' AMI programs and energy efficiency behavior-based programs, which could lead to double-counting of benefits.

26. It is reasonable to attempt to measure savings from certain behavior based programs.

27. The experimental design method, as described in the California Evaluation Protocols, and spelled out in greater detail by OPower's testimony, is well equipped to deal with certain analytical issues and policy concerns raised by the overlap of the savings targeted by comparative energy use reports, and programs already underway through Commission directive.

28. The policy determination in D.09-09-047 to freeze ex ante values could potentially lock in overly optimistic projections for behavior-based programs.

29. Results from the final 2006-2008 evaluation reports can be used as inputs for calculating the energy impacts of 2009 programs for those programs that were evaluated during the 2006-2008 period and also extended during 2009. Using these results would conserve resources and ensure consistency with the evaluation of the 2006-2008 programs from which the 2009 bridge-funding portfolios were derived.

30. Concerns about counting of Codes and Standards savings pre-2006 have been sufficiently resolved to allow 100% of savings to be counting toward energy savings goals.

Conclusions of Law

1. Proper notice was given to parties that the Commission may in this proceeding adopt changes that would modify D.05-01-055.

2. The Joint IOU/ED EM&V Plan is reasonable and should be adopted.

3. Experience with EM&V since D.05-01-055 requires that the Commission strengthen ED's management role for EM&V in order to minimize conflicts of interest, reduce duplication, and ensure transparency of information.

4. D.05-01-055 should be modified to provide specific oversight responsibilities for ED's management role for EM&V.

5. ED's management role for EM&V should include timely responses to IOU requests, including specific timeframes.

6. A budget of $125 million for EM&V for 2010 through 2012 is reasonable.

7. OP 42 of D.09-09-047 should be corrected to use the current program funding proportions for the 2010 through 2012 energy efficiency program cycle to determine EM&V funding for each utility.

8. $18.75 million of the $125 million EM&V budget for 2010 through 2012 should be allocated to the IOUs to maintain staffing levels.

9. It is reasonable to expect that IOUs should perform some or all of the types of EM&V studies that they have performed in the past.

10. $34.3 million of the $125 million EM&V budget for 2010 through 2012 should be allocated to the IOUs to perform EM&V studies (including staff costs).

11. It is reasonable for disputes regarding complex and controversial EM&V matters to be considered by the formal decision-makers in the Commission, after attempts at informal resolution.

12. The ED's recommendations as laid out in part of Section C of the ED Straw Proposal for IOU involvement in ED EM&V studies are reasonable.

13. It is reasonable to delegate to ED the task of determining the level of public vetting for EM&V projects.

14. ED's recommendation that all EM&V-related projects, regardless of funding source, adhere to the same policies and procedures as EM&V funded projects is reasonable, with the caveat that the EM&V processes should not apply to projects not previously considered to be in the EM&V category.

15. While ED and the IOUs should work with customers on evaluation surveys to both ensure necessary cooperation and to limit the burden on customers, there is no need to change any specific policy in this area.

16. It is reasonable to allow case-by-case exceptions to the Commission's firewall policy adopted in D.05-01-055 in order to recruit program implementers to collect data needed for EM&V.

17. It is reasonable to adopt a new set of electric and gas avoided costs for energy efficiency resources, including an updated GHG adder of $30/tonne in 2009 using generation cost inputs from the most recently adopted Market Price Referent as of the date of this Decision, and updating the natural gas cost forecast using data as of the date of this Decision.

18. It is reasonable to credit savings from certain proven behavior-based programs, using methodologies contained within the California Evaluation Protocols.

19. It is reasonable to use the results from the final 2006-2008 evaluation reports as inputs for calculating the energy impacts of 2009 programs, for those measures and programs that were evaluated during the 2006-2008 period and also extended during 2009.

20. 100% of pre-2006 Codes and Standards savings should be counted toward IOU energy savings goals.

ORDER

IT IS ORDERED that:

1. Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company shall adhere to the "2010 - 2012 Joint Energy Division and IOU Evaluation Measurement and Verification Plan" in Attachment A.

2. The budget of $125 million for Evaluation, Measurement and Verification (EM&V) for 2010 through 2012, tentatively adopted in Decision 09-09-047, is affirmed. A party seeking to increase the 2010-2012 EM&V budget may file a Motion in Rulemaking 09-11-014, the open energy efficiency Rulemaking.

3. Ordering Paragraph 42 of Decision 09-09-047 is corrected to read: "An initial Evaluation, Measurement and Verification budget of $125 million is adopted, subject to review in the follow-up Evaluation, Measurement and Verification decision in this docket. $88 million in remaining funds shall be used for these purposes, with $37 million in additional funds approved for 2010-2012. Evaluation, Measurement and Verification funds shall be allocated as follows: Pacific Gas and Electric Company 43%; Southern California Edison Company 39%; San Diego Gas & Electric Company 9%; and Southern California Gas Company 9%."

4. The process for Evaluation, Measurement and Verification for the 2010 through 2012 energy efficiency portfolios adopted in Decision 09-09-047 for Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company (collectively, IOUs) shall supersede the process adopted in Decision 05-01-055 regarding the following processes:

· An IOU shall seek approval from Energy Division before initiating Evaluation, Measurement & Verification (EM&V) ex-ante studies, or EM&V process or formative evaluations. The IOU management role for developing ex-ante savings estimates or EM&V process or formative evaluations shall be under the oversight of Energy Division, who shall have the authority to deny approval of projects. This authority is limited to situations where there is a conflict of interest with a contractor the IOU wishes to hire, where there is duplication or significant overlap with studies already planned or carried out by Energy Division, or where Energy Division can specify why a study is unnecessary. Energy Division's approval process for IOU's ex-ante studies, or EM&V process or formative evaluations, is limited to no more than two weeks. Any Energy Division denial of approval shall be in writing to the IOU requesting approval.

· If Energy Division expects to take three months or more to complete an ex ante estimate study, Energy Division shall approve an IOU request to develop ex-ante estimate in order to ensure timely information, or reject the request by providing the IOU, within two weeks of the IOU's request, with a written statement indicating that such rejection is due to duplication, conflict of interest or other specific rationale.

· Review of completed IOU workpapers regarding ex-ante savings estimates are subject to Energy Division review and approval, as set forth in an Administrative Law Judge Ruling of November 18, 2009 in Application 08-07-021, et al. Each IOU shall cooperate with Energy Division to allow upfront consultation regarding such workpapers.

· Energy Division's role for approval and involvement in IOU EM&V projects shall be as set forth in Attachment 2 of this decision.

· Energy Division may make case-by-case exceptions to the Commission-adopted firewall policy regarding program implementers in order to collect data needed for EM&V.

5. Decision (D.) 06-06-063 is modified to adopt a new set of electric and gas avoided costs for energy efficiency resources, including a greenhouse gas adder of $30/tonne, using generation cost inputs from the most recent Commission-adopted Market Price Referent as of the date of this order. Energy Division shall update the natural gas avoided cost for energy efficiency resources using natural gas price data as of the date of this order.

6. A total of $34.3 million for Evaluation, Measurement and Verification (EM&V) studies for 2010 through 2012, including staffing costs, allocated among Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company consistent with Ordering Paragraph #3 of this order. Following the process in Attachment 2 to this order, Energy Division shall determine if other EM&V funds shall be allocated to these utilities.

7. A party to Rulemaking (R.) 09-11-014 may file a "Motion for Evaluation, Measurement and Verification Dispute Resolution" (EM&V Motion) with the assigned Administrative Law Judge for resolution of an EM&V matter. The EM&V Motion must include a statement from Energy Division giving its side of the dispute and documentation of an attempt at informal dispute resolution. The Administrative Law Judge may issue a Ruling to resolve the dispute.

8. In a Motion for Evaluation, Measurement and Verification Dispute Resolution filed pursuant to Ordering Paragraph 7 of this order, the filing party or the Energy Division may ask that the matter be resolved by the assigned Commission or the full Commission. In that case, the Administrative Law Judge (ALJ) will consult with the assigned Commissioner to determine the appropriate course of action. In this situation, the assigned Commissioner or ALJ may issue a Ruling to resolve the dispute. If the assigned Commissioner determines the matter should be brought before the full Commission, the ALJ or assigned Commissioner shall issue a Proposed Decision and allow for comment under Rule 14 of the Commission's Rules of Practice and Procedure.

9. A Motion for Evaluation, Measurement and Verification (EM&V) Dispute Resolution filed pursuant to Ordering Paragraph 7 of this order may be used for the following purposes only:

· Dispute over selection of an EM&V contractor;

· Disputes about project-specific final EM&V work plans;

· Disputes over results of EM&V studies or reports (except for Energy Division Verification Reports, which are issued via draft resolutions per D.08-12-059);

· Disputes regarding final EM&V technical reports; and

· Disputes concerning public vetting of EM&V projects.

10. The process for Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company (investor-owned utilities or IOUs) involvement in Energy Division (ED) Evaluation, Measurement and Verification (EM&V) studies shall supersede the process adopted in Decision 05-01-055, and shall be as follows:

· ED and the IOUs will convene publicly-noticed meetings among their staff, EM&V contractors, and stakeholders to share key results and EM&V findings that might lead to improvements in the portfolio and identify best practices and possible improvements to evaluation methods. Such meetings will take place sometime around the middle of the program cycle or at such time when significant results from various EM&V projects are available. If so requested by parties or stakeholders, ED or IOUs, or both, should hold short informal meetings with groups or individual organizations, to discuss EM&V work progress and results.

· ED and IOUs will convene ad hoc meetings (approximately quarterly) among ED staff, EM&V contractors, IOU EM&V staff and IOU program managers to discuss work progress and results. These meetings are to provide for timely feedback to program design and implementation. The IOUs can request meetings with ED to discuss work progress and results at any time.

· When significant results are produced by the EM&V work, and a technical report is not immediately pending, the ED and/or the IOUs will provide informal written summaries of the results to the IOUs and other stakeholders. These written summaries will be posted on the same website used for posting EM&V work plans and comments.

11. Energy Division shall determine which Evaluation, Measurement and Verification projects should be publicly vetted, and shall follow the process laid out in the Energy Division Straw Proposal, pages 8-11, issued by Ruling in this proceeding on July 7, 2009.

12. All Evaluation, Measurement and Verification (EM&V) - related projects undertaken by Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company, regardless of funding source, shall adhere to the same policies and procedures adopted in this Order as EM&V-funded projects, except that such EM&V policies and procedures do not apply to projects not previously considered to be in the EM&V category.

13. Savings from behavior-based energy efficiency programs, defined as comparative energy use reporting contemplated in Senate Bill 488, shall be eligible for counting, if evaluated consistent with experimental design methods contained within the California Evaluation Protocols. The prioritization process described in the Joint Energy Division/Utility Plan for energy efficiency Evaluation, Measurement & Verification in 2010 through 2012, adopted in Ordering Paragraph 1 of this decision, shall be used to make decisions regarding which programs will be evaluated and specifically how those programs will be evaluated.

14. Savings for behavior-based energy efficiency programs shall be credited solely on an ex post basis.

15. Results from the final 2006-2008 evaluation reports shall be used as inputs for calculating the energy impacts of 2009 programs, for those measures and programs that were evaluated during the 2006-2008 period and also extended during 2009. The energy impacts of the 2009 programs shall be reported by Energy Division before the end of 2010, or as otherwise required in Rulemaking (R.) 09-11-014, R.09-01-019, or other applicable energy efficiency docket.

16. 100% of pre-2006 verified savings from Codes and Standards advocacy work shall count toward achievement of Commission energy savings goals for the 2010 through 2012 energy efficiency program cycle. Verified Codes and Standards savings pre and post-2006 shall count only for savings within the utility's service territory.

17. Applications (A.) 08-07-021, A.08-07-022, A.08-07-023, and A.08-07-031 are closed.

This order is effective today.

Dated April 8, 2010, at San Francisco, California.

Commissioners

I will file a concurrence.

/s/ TIMOTHY ALAN SIMON
Commissioner

ATTACHMENT 1

2010 - 2012 Joint Energy Division and IOU Evaluation Measurement and Verification Plan

1. Introduction and Scope of Joint EM&V Plan

Energy Division (ED) and the Investor-Owned Utilities (IOUs) submit this Joint Evaluation, Measurement and Verification (EM&V) / Policy and Planning (PP)23 Plan ("EM&V Plan") pursuant to Commission Decision 09-09-047, "Decision Approving 2010-2012 Energy Efficiency Portfolios and Budgets," issued on October 1, 2009. The EM&V Plan represents a cooperative effort by ED, Southern California Edison (SCE), Southern California Gas (SCG), San Diego Gas and Electric (SDG&E), and Pacific Gas and Electric (PG&E) staff to present a joint EM&V planning proposal and budget for the 2010-2012 energy efficiency portfolios authorized in Decision 09-09-047.

In Decision 09-09-047, the California Public Utilities Commission ("CPUC" or "Commission") addressed certain EM&V issues and policies and deferred resolution of others to a subsequent EM&V Decision. In anticipation of the subsequent EM&V decision, the Commission ordered ED and the IOUs to prepare an EM&V plan to be jointly submitted to the assigned ALJ and issued for comment via ruling. The plan presented herein is responsive to the Commission's stated desire "...to make near-term improvements in order to streamline EM&V processes, and enhance timeliness, transparency and consistency across EM&V work products" (D. 09-09-047 at p. 301) and "to take a fresh look at several aspects of our EM&V activity in California for the upcoming program cycle, to reduce unnecessary burden on staff and other resources, and streamline our EM&V processes." (D.09-09-047 at p. 294).

In D.09-09-047, the Commission adopted "1) a budget for 2010-2012 EM&V, 2) Commission goals for EM&V, and 3) a process for adopting detailed EM&V projects, refined EM&V budgets, and remaining EM&V policy issues in a subsequent EM&V Decision expected in the final quarter of 2009." The Commission clarified that the subsequent EM&V decision would include, but would not be limited to the following issues:

This EM&V Plan presents the ED and IOU joint proposals with respect to EM&V planning and budgeting.

2. Guiding Principles

The EM&V plan is guided by the Commission's Goals for EM&V, as articulated in Decision 09-09-047:

"EM&V activities shall be planned and implemented to achieve the following core objectives in order to support the Commission's oversight function of ensuring the efficient and effective expenditure of ratepayer funds within the energy efficiency portfolios. All activities should be undertaken to meet the overarching goals of clarity, consistency, cost-efficiency, and timeliness.

The core objectives are:

1. Savings Measurement and Verification - Measurement and verification of savings resulting from energy efficiency measures, programs, and portfolios serve the fundamental purpose of developing estimates of reliable load impacts delivered through ratepayer-funded efficiency efforts. Measurement and verification work should reflect a reasonable balance of accuracy and precision, cost, and certainty, and be designed for incorporation into in procurement planning activities.

2. Program Evaluation - Evaluation of program-specific qualitative and quantitative measures, such as the program performance metrics discussed earlier in this decision and process evaluations, serves a key role in providing feedback for the purposes of improving performance and supporting forward-looking corrections to utility programs and portfolios. In order to maximize return on ratepayer dollars, program evaluations must be completed on a timeline which informs mid-course corrections and/or program planning for the following cycle.

3. Market Assessment - In a constantly evolving environment, market assessments are an essential EM&V product needed to set the baseline for strategic design and improvement of programs and portfolios. Saturation studies, surveys of emerging technologies and other such analyses which inform estimates of remaining program potential and forward-looking goal-setting are key aspects of market assessment.

4. Policy and Planning Support - Consistent with prior program cycles, it is essential to reserve funding to support overarching studies and advisory roles which support Commission policy goals. Over the last program cycle this has been inclusive of potential and goals studies, maintenance of DEER database, developing databases of best practices for program design and delivery, program design mix, and other means which support the Commission's oversight role, but do not fall under the core EM&V categories described above.

5. Financial and Management Audit - Supporting the Commission's oversight function of ensuring the efficient and effective expenditures of ratepayer funds within the utilities' energy efficiency portfolios is another objective of EM&V activities. Rigorous financial and management audits overseen by Commission staff will be critical in ensuring that the utilities' general and administrative costs, and other program expenditures are prudent and reasonable.

3. Informal Goals and Guiding Principles for Increased ED/IOU Collaboration

Staff from ED and all four IOUs met for fours days of working meetings to develop EM&V budgets and a joint EM&V plan. During these meetings we agreed that we needed to develop a more collaborative and transparent working relationship as an important step towards improving the EM&V process. "Collaboration" in this document is defined as IOU and ED staff working together on shared EM&V projects, as well as working on separate EM&V projects following mutually agreed upon standards for transparency, respect, and communication. We believe that this more collaborative process will result in greater cost-efficiencies, more reliable results, broader stakeholder buy-in, and fewer disputed issues.

The following are informal goals for the purpose of fostering a working relationship built upon mutual respect and transparency. They represent an informal Energy Division and IOU staff-level agreement on general principles to guide staff-level collaboration and interaction on EM&V projects. These are not intended to impose formal or specific obligations on the ED or the IOUs and do not define the formal division of EM&V roles and responsibilities.

4. EM&V Planning Framework

At the time of this EM&V Plan, the 2010 - 2012 IOU portfolios are just recently adopted by the Commission, and program plans are expected to undergo additional refinement over the next four to six months through the final stages of the program planning process and compliance filings ordered by Decision 09-09-047. Additionally, as the adopted EE portfolio is implemented, program plans will necessarily evolve to adapt to changing circumstances, program funding may be shifted around, new programs may be designed and fielded, and some programs may be terminated. For these reasons, ED and IOU staff have agreed that the optimal EM&V plan for the Commission to adopt at this time is an EM&V Planning Framework guided by existing Commission policy and ED & IOU staff experience and expertise that gives EM&V the needed flexibility, rather than a detailed plan that makes assumptions about the full scope of EM&V needs over a three year period. This document outlines the basic elements of this proposed EM&V Planning Framework, and ED/IOUs jointly request party input primarily on this proposed Framework and proposed areas of work, not on specific draft EM&V/PP project budgets.

Given the many Commission required EM&V projects, multiple possible EM&V needs, and constraints on EM&V staff and consulting resources, there is a need to prioritize and optimize across EM&V research areas and individual projects, as well as a need to plan and implement EM&V project in phases. To accomplish the next three years worth of EM&V as effectively and efficiently as possible, ED and IOU EM&V staff propose the following EM&V Planning Framework:

5. Initial EM&V Plan

As discussed above, ED and the IOUs plan to design and implement EM&V in phases by order of project priority. The First Phase of EM&V projects is work that needs to be immediately initiated in order to set up a more efficient EM&V "infrastructure" that makes cost-effective improvements on the use of all EM&V resources, data, and processes. The First Phase will also include some research projects that are immediately needed by the IOUs in order to make rapid adjustments to the new program portfolio. ED and the IOUs intend to begin work on projects in this First Phase using previously approved 2009 bridge-funding for EM&V. The First Phase projects will need to begin prior to a final Commission Decision approving EM&V plans and budgets, hoped for in late 2009. Several of the First Phase projects will be ongoing and may continue through 2012. The Second Phase and Third Phase projects are briefly describe at the end of section 5.
ED and the IOUs expect the First Phase to consist of the following projects:

A Second Phase of projects will be planned and implemented as soon as assignments are made and work is underway on the First Phase projects, but no later than the first quarter of 2010. The Second Phase projects will include the formative M&V, process evaluation, and market research that is needed to provide early assessments of the programs and make decisions about program modifications, but which were not launched as part of the First Phase. We anticipate that the Second Phase projects will be initiated during the first and second quarters of 2010.

Finally, a Third Phase of EM&V projects will be planned and implemented when ED and IOU staff are convinced that Second Phase projects are successfully underway and likely to achieve project goals. The Third Phase projects will primarily be the summative, or ex-post, evaluations that have been employed by the Commission to establish retrospective statements of portfolio accomplishments. Additional formative work may also be implemented during the Third Phase of EM&V projects, if needed. We anticipate that the Third Phase projects will be initiated between the second and fourth quarters of 2010, after the Commission rules on the incentive mechanism for 2010-2012 in Rulemaking 09-01-019.

6. Proposed EM&V Budget

Below we present the proposed allocation of authorized budget for all ED and IOU EM&V projects, as well as ED staff Policy and Planning projects. In D.09-09-047, the Commission indicated a desire to keep the EM&V budget at 4% (approximately $125 million) with the expectation that the ED and IOU EM&V staff can produce cost efficiencies and streamline the scope and reporting of EM&V projects. While the Commission also indicated a possibility that it would consider changes to the initial EM&V funding based on proposals for additional funding brought forth in the EM&V plan, ED and the IOU EM&V staff have taken the Commission's desire to manage costs seriously and will strive to complete a robust research portfolio for under $125 million.

While we are confident that the authorized budget will be sufficient to complete a reasonably comprehensive set of EM&V projects, the range of studies needed for 2010-2012 is substantially greater than the range of studies completed for 2006-2008. We are therefore compelled to emphasize that some potentially important research projects may not be implemented if we are to prioritize effectively. Thus, we ask that the EM&V decision keep open the option offered in D.09-09-047 to request more funding if we determine that sufficiently important projects cannot be funded.

2006-2008 were start-up years for both the ED and the IOU EM&V groups, with many start-up difficulties and new systems that did not function optimally. As a result, a number of the important planned studies could not be completed. The experiences of 2006-2008 uncovered some weaknesses in current utility and CPUC tools that need to be strengthened. These include the EM&V structure itself, utility tracking and reporting systems and their ability to meet EM&V data needs, as well as multiple concerns surrounding ex-ante savings parameter updating and documentation, and cost-effectiveness issues. Finally, the adoption of the California Long Term Energy Efficiency Strategic Plan pushes programs, planning, and coordination in far-reaching new directions. This necessitates a major investment in coordinating with market actors and state agencies, policy analysis, and planning for the 2013-2015 cycle and beyond, in each of the strategic areas of focus, and it creates a host of new information needs.

The IOU EM&V team solicited input from program staff regarding the programs that will be offered during the 2010-2012 cycle. The EM&V team also reviewed the process evaluations, market assessments, and early M&V projects performed during the 2006-2008 cycle to identify additional research requirements. In collaboration with the program staff, the EM&V team compiled a list of market assessment and early M&V needs and process evaluations related to the programs that are being offered in 2010-2012. Based on previous experience, the EM&V team then estimated the cost of performing these studies, including the costs related to the EM&V staff.


ED staff developed budgets for impact evaluations, performance metric evaluations, and overarching and support projects using expert judgment and experience managing similar projects during the 2006-2008 timeframe. These estimates take into consideration expected efficiencies to be gained from the proposed prioritization and optimization process, as well as the fact that the projects will be managed by staff (both IOU and ED) that have gained considerable additional experience managing the 2006-2008 EM&V projects.

The specific studies and their associated budgets listed in the Table in Section 6c below are ED and the IOU's current estimate of the optimal allocation of the authorized EM&V budget. Section 5 of this plan describes the multi-stage process that ED and the IOUs will go through for determining and prioritizing what studies will be done, when, and with what level of project budget. The process will include making decisions about which organization will contract for each project, who will take primary project management responsibility for it, and the level of involvement of the Commission staff in overseeing each project.

In order to allow ED and the IOUs to respond to changes in the market and to new insights in evaluation, fund-shifting flexibility is needed within the EM&V budget. This includes not only shifting funds between projects, but also, to some extent, between funds managed by the IOUs and those managed by ED, as they mutually agree. ED and the IOUs agree that a minimum allocation of 15% of the EM&V budget to the IOUs is appropriate to support necessary EM&V activities until such time as the Commission issues a final EM&V decision and budget. These costs are currently included as part of the process evaluation, market assessment and early M&V study costs in the budget estimates in Table C. ED and the IOUs were not able to reach consensus as to any further pre-allocation of the remaining 85% of the EM&V budget. ED and the IOUs agree that it is appropriate for the IOUs to include any specific proposals for allocation of the remaining 85% of the EM&V budget in their comments to this EM&V Plan. As during the 2006-2008 cycle, the utilities will be responsible to pay the Energy Division-approved costs for all projects contracted and/or managed by Energy Division.

Each utility's EM&V budget will be its proportional share of the total EM&V budget approved by the Commission, with the proportion equal to its proportion of total program budgets: 43% for PG&E; 39% for SCE; and 9% each for SDG&E and SoCalGas.

This requires correcting Ordering Paragraph 42 of D.09-09-047, which inadvertently used the program funding proportions from the 2006-2008 cycle.

Each utility will pay for its studies that are determined to be acceptable utility-specific studies, out of its overall EM&V budget.

This EM&V fund-shifting flexibility request is consistent with Commission practice for at least the last two decades, and probably for the entire history of EM&V funding for EE programs. The Commission has always recognized the benefits of setting an overall budget but allowing EM&V decision-makers to determine EM&V priorities and budget allocations for the costs to meet them in an ongoing process, rather than assuming that needs and priorities are all known in advance and will be unchanged over a program funding cycle.

The general EM&V project area descriptions below provide summaries of the categories of work used to set the EM&V budget proposed herein. While these project areas are considered necessary preliminarily, they are provided for illustrative purposes and are subject to change as ED and the IOUs continue with the prioritization process. Final research project goals, scope, timing, and deliverables will be determined during development of detailed statements of work included in the contracting process.

The Market Assessment studies that will be conducted by ED and the IOUs will include two different study types: market characterization and market baseline measurement.

Market Characterization is a quantitative and qualitative assessment of the structure and functioning of a market, the primary purpose of which is to understand key components and magnitudes of a market, and how the market operates. The study also provides information on how to effectively change the way in which the market functions.

Market Baseline Measurement is the quantification of key market indicators that have been or can be influenced by a program intervention. The primary purpose of the baseline measurement is to provide a basis for later comparisons of the status of the market after program intervention, in order to help assess the impact of the program. This study can also include quantification of size of a particular market so we can monitor the share of market as a result of program intervention.

Attachment 2

Process for Commission Oversight
of IOU EM&V Project Initiation

1. Project Formation: IOUs notify ED of their intention to conduct an EM&V project and solicit input from ED on the shaping of the project. ED may choose to waive this opportunity to participate if it chooses. The point of this step is to minimize potential delays in the following steps.

2. Project Description: Once the need for a project has been determined, the IOUs will prepare a project description (basically a high level scope of work, following reporting standards to be developed).

3. Project Tracking: The project description will be uploaded to the Energy Division's project tracking system.

4. Project Review and Approval: the project description will be available for review and approval by Energy Division for one calendar week.

5. Project Initiation: Once the ED review and approval is completed (or waived) the IOUs may begin implementing the project in accordance with the project description.

6. Project RFP and Proposals: If the project requires competitive bidding, the IOUs will upload the RFP to the Energy Division project tracking system. If the project involves consultant proposals, the proposals will be uploaded to the Energy Division project tracking system.

7. Contractor Selection: If the project involves hiring a contractor, whether by competitive or directed bid, the IOUs will notify ED of their preferred contractor and other contractors who were considered and/or who submitted bids. ED will make the final selection of all EM&V contractors.

Process for Commission oversight of IOU EM&V project implementation

1. Project Reporting: The IOUS will upload project documents to the Energy Division project tracking system. The required project documents and standards for timing will be determined at a later time. Project documents will include EM&V work plans, schedules, methodologies, analyses, draft reports, and interim findings.

2. Project Briefings: The IOUs will provide briefings on all EM&V projects to ED at regular intervals.

3. Project Advisory Meetings: Certain projects will be selected by ED as requiring the opportunity for regular ED participation. For these projects, the ED liaison will be notified of project meetings.

(END OF ATTACHMENT 2)

1. What are the respective roles of Energy Division and IOU EM&V staff for conducting EM&V projects?

i. Are the IOUs permitted to manage any impact evaluation or M&V projects that develop ex-ante savings estimates which may be used for determining portfolio performance, reporting accomplishments, or calculating incentives? If so, what are the Commission's expectations for rules and procedures for oversight of these projects?

Energy Division Recommendation - Question 1.i.
The IOUs should be permitted to manage projects to develop energy savings estimates in the specific case where there is no existing ex-ante estimate or the IOUs believe that an existing estimate is out of date and needs testing AND Energy Division is not already conducting or planning to conduct a project to develop estimates for the same measure. The IOUs should be required to seek approval from Energy Division before initiating such work and should proactively provide opportunities for Energy Division to review project milestones and provide input directly to the project manager. The Commission should clearly and explicitly authorize Energy Division to oversee such projects, including authorization to deny approval of projects that are not in the ratepayers interest. Energy Division's project approval will follow the process outlined in Energy Division's recommendations for questions 2,3, and 4.

ii. Is Energy Division expected and therefore permitted to initiate and manage evaluations that may be considered process or formative evaluations?

Energy Division Recommendation - Question 1.ii.
The Commission should clearly and explicitly authorize Energy Division to conduct any type of EM&V consistent with the following guidelines, which are adapted from the ED Straw Proposal Issued by ALJ Ruling on July, 7th, 2009 in A.08-07-021.

iii. Should ED have the authority to be involved in projects that develop ex-ante savings estimates, such as the non-DEER work papers, which are currently managed by the IOUs without any ED involvement?

Energy Division Recommendation - Question 1.iii.
The IOUs should be required to notify Energy Division of all workpaper development activities and should proactively provide opportunities for Energy Division to review methodologies and provide input to the workpaper authors. ED involvement at this stage will streamline the review of final workpapers and will ensure greater reliability of workpaper savings estimates. Energy Division's involvement in workpaper projects will follow the process outlined in Energy Division's recommendations for questions 2,3, and 4.

2. Should ED be responsible for approving IOU EM&V projects? Should there be exceptions to this process for expedited projects?

3. Current policy requires ED to approve all IOU EM&V contractors in order to manage contractor conflicts of interest. Should this process continue or be modified?

4. Should ED have the authority to be involved in IOU EM&V projects?

Energy Division Recommendation - Questions 2,3, and 4:
Energy Division proposes that its involvement in authorizing and reviewing IOU EM&V projects, including the ex-ante savings estimation projects discussed in 1.i. and 1.iii. above, be managed according to the following procedures, adapted from the ED Straw Proposal:

Process for Commission oversight of IOU EM&V project initiation

8. Project Formation: IOUs notify ED of their intention to conduct an EM&V project and solicit input from ED on the shaping of the project. ED may choose to waive this opportunity to participate if it chooses. The point of this step is to minimize potential delays in the following steps.

9. Project Description: Once the need for a project has been determined, the IOUs will prepare a project description (basically a high level scope of work, following reporting standards to be developed).

10. Project Tracking: The project description will be uploaded to the Energy Division's project tracking system.

11. Project Review and Approval: the project description will be available for review and approval by Energy Division for two calendar weeks.

12. Project Initiation: Once the ED review and approval is completed (or waived) the IOUs may begin implementing the project in accordance with the project description.

13. Project RFP and Proposals: If the project requires competitive bidding, the IOUs will upload the RFP to the Energy Division project tracking system. If the project involves consultant proposals, the proposals will be uploaded to the Energy Division project tracking system.

14. Contractor Selection: If the project involves hiring a contractor, whether by competitive or directed bid, the IOUs will notify ED of their preferred contractor and other contractors who were considered and/or who submitted bids. ED will make the final selection of all EM&V contractors.

Policy Issue #3 (part of C. in ED Straw Proposal)

Process for Commission oversight of IOU EM&V project implementation

4. Project Reporting: The IOUS will upload project documents to the Energy Division project tracking system. The required project documents and standards for timing will be determined at a later time. Project documents will include EM&V work plans, schedules, methodologies, analyses, draft reports, and interim findings.

5. Project Briefings: The IOUs will provide briefings on all EM&V projects to ED at regular intervals.

6. Project Advisory Meetings: Certain projects will be selected by ED as requiring the opportunity for regular ED participation. For these projects, the ED liaison will be notified of project meetings.

Finally, ED will exercise the authority granted to Commission staff under Public Utilities Code Section 314 (a), as needed, to review process evaluation plans and results.

5. Should ED have the authority to allocate the authorized EM&V budget between ED and IOU managed EM&V projects according to the overall EM&V priorities?

Energy Division Recommendation - Question 5:
The Commission should grant ED authority to approve IOU projects as discussed in the recommendation on questions 2,3, and 4 above. With this authority and the adoption of the prioritization process discussed in the Joint IOU/ED EM&V Plan, ED believes that a specific prior allocation to IOU managed projects above and beyond the 15% minimum to fund EM&V staff is unnecessary. Nevertheless, ED anticipates that the IOUs will request, and are likely to be granted responsibility to manage a sizable share of the EM&V work.

ED believes that the intention of the following statement on page 301 of Decision 09-09-047, "EM&V plans and budgets for 2010-2012 should be categorized in accordance with the first four objectives articulated above, and will be prioritized for approval in following with the most pressing needs across each category" is to allocate EM&V resources according to overall research priorities, rather than across organizations responsible for implementing EM&V projects.

6. How should major disputes arising out of the EM&V work be managed? When should these disputes be elevated to the full Commission for resolution?

Energy Division Recommendation - Question 6:
Energy Division maintains its recommendations articulated in the ED Straw Proposal, excerpts reproduced below:

Project-Specific EM&V Plans:

If parties continue to take issue with the final work plans, a party or parties may file a motion with the Assigned ALJ and provide evidence for why the plans should be changed and how. The ALJ will resolve the dispute and direct Energy Division and/or the IOUs to revise the plans accordingly via ruling.

EM&V Technical Reports

If parties continue to take issue with the final EM&V technical reports, a party or parties may file a motion with the Assigned ALJ and provide evidence for why the report is deficient and what changes to the report would be necessary to correct the deficiency. The ALJ will resolve the dispute and direct Energy Division and/or the IOUs, via ruling, to prepare an addendum to the report correcting the deficiency. The addendum will be posted on the same website where the draft reports are posted.

7. How extensively should IOUs be involved in ED EM&V projects?

Energy Division Recommendation - Question 7:
Energy Division maintains its recommendations articulated in Section C of the ED Straw Proposal "Stakeholder Input Process and Approval of EM&V Projects," as well as the informal interactions proposed in the Joint IOU/ED EM&V Plan.

8. What is the appropriate level of public involvement in EM&V projects? Should certain EM&V project be exempted from a full public process? How will the exempted EM&V projects be determined?

Energy Division Recommendation - Question 8:
In their comments on the ED Straw Proposal, the IOUs expressed concerns that engaging with the public on every EM&V project, as proposed in the ED Straw Proposal, would be ineffective and would slow down the implementation of time-sensitive projects. At least one utility has proposed that projects under a specific budget should be exempt from the type of public process proposed in Section C of the ED Straw Proposal and that the key stakeholders for process evaluations are limited to program administrators and implementers. ED believes that there will be IOU EM&V projects that will not require an intensive public vetting process, but we do not believe the project budget is a reasonable indicator of the need for public vetting. Additionally, ED strongly believes that ratepayers and the CPUC are in fact key stakeholders for process evaluations. To ensure that the appropriate EM&V projects are publically vetted and that time-sensitive projects are not delayed, ED recommends that the Commission grant ED authority to determine which EM&V projects should and should not undergo public vetting.

9. Should all IOU EM&V related projects, regardless of funding source (such as projects that develop savings estimates for non-DEER measures funded out of program funds), be required to follow the same policies and procedures that are required for EM&V funded projects?

Energy Division Recommendation - Question 9:
Energy Division recommends that the Commission clearly require, without exception, that all EM&V related projects, regardless of funding source, be required to adhere to the same policies and procedures as EM&V funded projects.

10. Should the IOUs modify program eligibility rules to require very large customized program participants to participate in evaluations if selected in a sample, as a condition for receiving EE funding?

Energy Division Recommendation - Question 10:
Many large project participants have either refused to participate in evaluations or have Energy Division believes it is reasonable to require participants who receive a large sum of EE funding and services to participate in evaluations, if needed. This participation would include on-site measurement and verification, as well as surveys of key participant personnel. Energy Division proposes to review past projects with the IOUs to determine the EE incentive threshold above which participation in evaluations would be obligatory.

Energy Division will endeavor to reduce the burden of participating in evaluations by coordinating with the IOU implementation and inspection process. Energy Division recommends that the Commission require the IOUs to cooperate with ED in this regard.

11. Should the Commission allow the IOUs the opportunity to count savings from behavior based programs?

i. How should the Commission develop EM&V methodologies to verify savings driven by behavior-based efficiency programs?

ii. What analytical issues are raised by changing policy to allow credit and require measurement of savings driven by behavior-based efficiency programs (i.e. savings persistence, potential double-counting of savings by other resource programs, potential double-counting of savings claimed as part of the conservation benefits assumed to underlie Advanced Meter Infrastructure (AMI) business cases [PG&E - D.09-03-026; SCE - D.08-09-039; SDG&E - 07-04-043])?

Energy Division Recommendation - Question 11:
Energy Division believes that the categories of behavior based programs need to be more specifically defined and measurement issues need to be clarified before categorically recommending savings credit from behavior based programs. Energy Division believes there are significant intersecting issues with the IOUs' AMI programs. For instance, we believe it is the intent of the AMI program to provide customers with usage data to help them manage their energy consumption through conservation. Comparative usage reporting and benchmarking could be provided as part of the bundle of AMI services and may thus be considered AMI generated conservation savings. ED will evaluate the IOUs comparative usage programs using experimental design consistent with SB 488. ED proposes to refine the reporting required of IOUs by SB 488 with IOU input. Measuring and quantifying savings from other behavior based programs, such as Marketing, Education, and Outreach may be complicated and developing protocols for this measurement approach may take some time.

Energy Division recommends that the Commission consider forming a working group, facilitated by Energy Division, to explore these issues.

(END OF ATTACHMENT 3)

Concurrence of Commissioner Timothy Alan Simon

Decision Determining Evaluation, Measurement, and Verification

Processes for 2010 Through 2012 Energy Efficiency Portfolios

A.08-07-021/D.10-04-029

Effective Evaluation, Measurement, and Verification (EM&V) is absolutely critical to the success of our Energy Efficiency programs in measuring peak load savings and the cost-effectiveness of our programs and portfolios. I support this Decision as it seeks to increase transparency, efficiency, and accountability in our EM&V processes while minimizing conflicts of interest. I am optimistic that the dispute resolution process adopted by this Decision will result in greater collaboration between program implementers and evaluators.27

I am also very supportive of the Decision's clarification of contracting processes for EM&V consultants. Maximizing competition for contracts in EM&V and other energy efficiency program evaluation and implementation activities will ultimately result in greater cost-effectiveness of our programs and portfolios. I recognize that expertise in EM&V may be somewhat limited, but I believe we should conduct small business outreach to promote supplier diversity in this area.

With a $125 million approved EM&V budget for 2010 through 2012, I expect the coordination efforts ordered by this Decision to be taken seriously so as to avoid duplication of data collection and evaluation efforts.28 I look forward to monitoring our progress through this program cycle to ensure that we are adhering to performance metrics, improving outreach and procurement for EM&V contract opportunities, and getting the most out of the ratepayer dollars in our EM&V budget.

Dated April 20, 2009, at San Francisco, California.

23 Throughout this document "EM&V," if not otherwise indicated, is understood to refer to both traditional Evaluation, Measurement and Verification work as well as the Energy Division's policy and planning activities. (See Decision 09-09-047 pages 295-298 and 387 for more on policy and Planning as included in the EM&V budget and activities).

24 http://www.eval.org/publications/guidingprinciples.asp.

25 http://www.nspe.org/Ethics/CodeofEthics/index.html.

26 http://www.ashrae.org/publications/detail/16451.

27 Decision Determining Evaluation, Measurement, and Verification Processes for 2010 Through 2012 Energy Efficiency Portfolios (D.10-04-029), March 9, 2010 at 29-34.

28 Id. at 7.

Previous PageTop Of PageGo To First Page