ALJ/DMG/avs DRAFT Agenda ID #9278 (Rev. 1)
Ratesetting
4/8/2010 Item 52
Decision PROPOSED DECISION OF ALJ GAMSON (Mailed 3/9/2010)
BEFORE THE PUBLIC UTILITIES COMMISSION OF THE STATE OF CALIFORNIA
Application of Southern California Edison Company (U338E) for Approval of its 2009-2011 Energy Efficiency Program Plans and Associated Public Goods Charge (PGC) and Procurement Funding Requests. |
Application 08-07-021 (Filed July 21, 2008) |
And Related Matters. |
Application 08-07-022 Application 08-07-023 Application 08-07-031 |
DECISION DETERMINING EVALUATION, MEASUREMENT
AND VERIFICATION PROCESSES FOR 2010 THROUGH
2012 ENERGY EFFICIENCY PORTFOLIOS
TABLE OF CONTENTS
Title Page
DECISION DETERMINING EVALUATION, MEASUREMENT AND VERIFICATION PROCESSES FOR 2010 THROUGH 2012 ENERGY EFFICIENCY PORTFOLIOS 22
2.1. Energy Division Straw Proposal 55
2.2. Joint Energy Division/Utility EM&V Plan (Joint Plan) 66
3. EM&V Budget for 2010 Through 2012 77
4. The Joint Energy Division /Utility EM&V Plan (Joint Plan) 1212
5. Respective Roles and Responsibilities of ED and IOUs 1414
5.1. IOU-Managed Impact Evaluations (DEER) 1515
5.2. ED Managed Process Evaluation 1717
5.3. ED-Involvement in IOU Workpaper Development (non-DEER) 1919
5.4. IOU Involvement in ED Projects 2121
5.5. ED Review of all IOU EM&V Products 2323
5.6. ED Role in IOU Process Evaluations and Markets Assessments 2626
6. Dispute Resolution Mechanism 2929
7. Customer Participation in EM&V as a Condition of EM&V Funding 3434
8. Counting Savings from Behavior Based Programs 3636
9. Other Issues Deferred by D.09-09-047 4242
9.1. EM&V Contractor Firewall Issues 4242
9.3. 2009 Bridge Funding Reporting, Budget Allocation, and EM&V 4444
Title Page
ATTACHMENT 1 - 2010-2012 Joint Energy Division and IOU Evaluation
Measurement and Verification Plan.
ATTACHMENT 2 - Process for Commission Oversight of IOU EM&V
Project Initiation
ATTACHMENT 3 - Energy Division EM&V Questions and Recommendations
from November 2009 ALJ Ruling
DECISION DETERMINING EVALUATION, MEASUREMENT
AND VERIFICATION PROCESSES FOR 2010 THROUGH
2012 ENERGY EFFICIENCY PORTFOLIOS
1. Summary
This decision sets out the roles and relationships among the Commission's Energy Division (ED), California's investor-owned utilities (utilities or IOUs), and stakeholders regarding Evaluation, Measurement and Verification (EM&V) of energy efficiency programs for 2010 through 2012. The roles and responsibilities previously laid out in Decision (D.) 05-01-055 are clarified to improve transparency of EM&V activities, minimize conflicts of interest, and reduce duplication of effort and undue expenditure of ratepayer funds for the 2010 through 2012 time period.
Credible and effective EM&V requires a clear separation between "those who do" (the program administrators and implementers) and "those who evaluate" the program performance. Accordingly, we do not alter the fundamental division of responsibilities struck in D.05-01-055, under which the ED maintains management and contracting responsibilities for all EM&V studies used to measure and verify energy, peak load savings and cost-effectiveness for individual programs, groups of programs and at the portfolio level, while the utilities retain a limited EM&V budget to carry out studies that inform portfolio implementation and process evaluation. On the basis of experience over the past several years of EM&V activity, however, we make several process changes to improve oversight and accountability of EM&V activities carried out by both ED and the utilities.
All parties agree that we need to codify more collaborative and transparent processes as an important step towards improving the effectiveness of our EM&V efforts. In this decision we set forth new standards for transparency, coordination, and stakeholder engagement relating to EM&V projects carried out by both the utilities and ED. We believe that this more collaborative process will result in greater cost-efficiencies, more reliable results, broader stakeholder buy-in, and fewer disputed issues. In particular, we:
· Clarify process for ED review of all IOU EM&V contracting decisions;
· Grant IOU authority to develop ex ante values under limited circumstances;
· Grant ED authority to conduct process evaluations;
· Clarify process for stakeholder input on all EM&V projects; and
· Provide a new resolution process for disputes over EM&V processes and findings.
This decision finalizes the $125 million budget for EM&V activity over 2010-2012. We also approve a Joint Plan submitted by the utilities and ED, which lays out a roadmap for the EM&V studies to be performed on the 2010-2012 energy efficiency portfolios approved in D.09-09-047.
Finally, we address certain carryover policy issues, including the treatment of savings estimates from behavioral programs and codes & standards.
2. Background
The crux of the success of energy efficiency as California's resource of first choice lies in evaluation, measurement and verification (EM&V). EM&V is important for several reasons. First, it is necessary to determine whether and how well current individual programs are working. Second, EM&V is critical in considering how to improve programs and for development of new measures. Third, EM&V is used on a broad level to measure whether the investor-owned utilities (IOUs) are meeting, on a portfolio basis, the overall energy savings goals established by the Commission.1 Fourth, EM&V results are used to determine whether IOUs should receive rewards or pay penalties as part of the Risk Reward Incentive Mechanism (RRIM) adopted by the Commission.2 Fifth, robust EM&V is critical to ensure that the IOUs and the state can depend on energy efficiency as a resource.
Decision (D.) 05-01-055 returned the state's IOUs to the role of energy efficiency program administrators. That decision provided direction on how EM&V should be structured after 2005. In particular the decision found that credible and effective EM&V required a clear separation between "those who do" (the program administrators and implementers) and "those who evaluate" the program performance. Accordingly, the decision assigned to the Commission's Energy Division (ED) management and contracting responsibilities for all EM&V studies that will be used to (1) measure and verify energy and peak load savings for individual programs, groups of programs and at the portfolio level, (2) generate the data for savings estimates and cost-effectiveness inputs, (3) measure and evaluate the achievements of energy efficiency programs, groups of programs and/or the portfolio in terms of the "performance basis" established under Commission-adopted EM&V protocols and (4) evaluate whether programs or portfolio goals are met.
In recognition that IOU portfolio managers and program implementers need access to market information to perform their responsibilities, D.05-01-055 provided that the IOUs could manage "a limited subset of evaluation studies as long as there is no potential for conflict due to the nature of the study, and as long as ED makes the final selection of contractors."
As a further safeguard to ensure against conflict-of-interest in EM&V, D.05-01-055 prohibited entities from performing these types of EM&V studies at the same time they are under contract for program delivery work -- either as a non-IOU program implementer or subcontractor to an IOU implementer.
On July 7, 2009, an Administrative Law Judge (ALJ) Ruling sought comment on an ED "Straw Proposal" on EM&V issues for the 2010-2012 program cycle. The Ruling asked a number of questions about issues discussed in the Straw Proposal, including potential modifications to the overall goals of EM&V, respective scopes of EM&V responsibilities for Commission and utility staff, stakeholder input process and approval of EM&V projects, and several other issues.
Parties commented on July 27, 2009. In the July 7 Ruling, parties in this proceeding were given notice that the Commission may in this proceeding adopt changes that would modify D.05-01-055. To that end, the July 7 Ruling with its attachments was served on the service list in R.01-08-028 (the proceeding in which D.05-01-055 was issued).
In D.09-09-047, the Commission adopted energy efficiency portfolios for 2010 through 2012. The decision addressed certain threshold issues pertaining to EM&V issues for the 2010-2012 portfolios, including: 1) a preliminary budget for 2010-2012 EM&V of 4% of total energy efficiency expenditures or $125 million for 2010 -2012, 2) Commission core objectives for EM&V; and 3) a process for adopting detailed EM&V projects, refined EM&V budgets, and remaining EM&V policy issues in a subsequent EM&V Decision.
In anticipation of this subsequent EM&V decision, the Commission in D.09-09-047 ordered Energy Division and the IOUs to prepare an EM&V plan (the Joint Plan) to be jointly submitted to the assigned ALJ and issued for comment via Ruling. The Joint Plan stated that it was responsive to the Commission's stated desire "...to make near-term improvements in order to streamline EM&V processes, and enhance timeliness, transparency and consistency across EM&V work products" (D.09-09-047 at 301) and "to take a fresh look at several aspects of our EM&V activity in California for the upcoming program cycle, to reduce unnecessary burden on staff and other resources, and streamline our EM&V processes." (D.09-09-047 at 294.) However, ED and the IOUs were unable to agree on a number of items related to the EM&V plan.
In a November 20, 2009 ALJ Ruling, parties were asked to respond to a number of questions posed by ED on EM&V issues which were not resolved as part of the Joint Plan. The Joint Plan was presented in Attachment 1 to the ALJ Ruling. Attachment 2 to the ALJ Ruling listed a number of questions that need to be considered in the upcoming EM&V decision, along with recommendations of ED. The questions and ED recommendations are reprinted in Attachment 3 to this decision for reference.
D.09-09-047 also deferred other issues to a subsequent decision. Comments were filed on these issues in July and August 2009. In this order, we take up outstanding issues including: contractor firewalls; 2009 avoided costs; 2009 bridge funding reporting, budget allocation and EM&V; and Codes and Standards.
3. EM&V Budget for 2010 Through 2012
We approved a budget of $125 million, or 4% of the overall portfolio budgets, for 2010 through 2012 EM&V in D.09-09-047, subject to review in this decision. This preliminary budget reflected the expectation that, drawing from the experience of EM&V over the past program cycle, ED and IOU EM&V staff can produce cost efficiencies and streamline the scope and reporting of EM&V projects for 2010-2012.
In the Joint Plan, ED and the IOUs state that they have taken the Commission's desire to manage costs seriously and will strive to complete a robust research portfolio for under $125 million. However, the Joint Plan notes that the range of studies needed for 2010 through 2012 is substantially greater than the range of studies completed for 2006-2008, and thus asks that the EM&V decision keep open the option offered in D.09-09-047 to request more funding if we determine that sufficiently important projects cannot be funded. No party suggested any change to the overall EM&V budget.
We hereby finalize the overall budget level of $125 million for 2010 through 2012 tentatively adopted in D.09-09-047. If parties seek to increase the 2010-2012 EM&V budget, they may file a Motion in R.09-11-014, the open energy efficiency Rulemaking. The assigned ALJ and/or assigned Commissioner may rule on such a Motion or may prepare a Proposed Decision (PD) for full Commission consideration.
In the Joint Plan, ED and the IOUs recommend that each utility's EM&V budget should be its proportional share of the total EM&V budget approved by the Commission, with the proportion equal to its proportion of total 2010 through 2012 program budgets: 43% for Pacific Gas and Electric Company (PG&E); 39% for Southern California Edison Company (SCE); and 9% each for San Diego Gas & Electric Company (SDG&E), and Southern California Gas Company (SoCalGas). We adopt this recommendation. This allocation requires correcting Ordering Paragraph (OP) 42 of D.09-09-047, which inadvertently used the program funding proportions from the 2006-2008 cycle.
3.1. Allocation for IOU EM&V Activities
As noted above, our framework provides that the majority of the EM&V budget will be for studies managed by the Energy Division. However, a limited number of studies, pursuant to D.05-01-055 and the direction we give today, will be carried out by the IOUs. An important question is whether this decision should allocate a specific portion of the EM&V budget (set at $125 million as determined above) to the IOUs and, if not, what will be the process and, in particular the role of the Energy Division, in deciding the IOU budget. This issue was raised in Question 5 of the November 20, 2009 Ruling which asked: "Should ED have the authority to allocate the authorized EM&V budget between ED and IOU managed EM&V projects according to the overall EM&V priorities?"
In the Joint Plan at 18, ED and the IOUs agreed that a minimum allocation of 15% of the EM&V budget to the IOUs is appropriate to maintain and support necessary EM&V activities until such time as the Commission issues a final EM&V decision and budget. The Joint Plan noted that these costs are currently included as part of the process evaluation, market assessment and early M&V study costs in the budget estimates in Table C of the Joint Plan. ED and the IOUs were not able to reach consensus as to any further pre-allocation of the remaining 85% of the EM&V budget.
ED recommends that the Commission grant it authority to approve IOU projects. With this authority and the adoption of the prioritization process discussed in the Joint Plan, ED believes that a specific prior allocation to IOU managed projects above and beyond the 15% minimum to fund EM&V staff is unnecessary. Nevertheless, ED anticipates that the IOUs will request, and are likely to be granted, responsibility to manage a sizable share of the EM&V work.
ED believes that the intention of the following statement on page 301 of D.09-09-047, "EM&V plans and budgets for 2010-2012 should be categorized in accordance with the first four objectives articulated above, and will be prioritized for approval in following with the most pressing needs across each category" is to allocate EM&V resources according to overall research priorities, rather than across organizations responsible for implementing EM&V projects.
Division of Ratepayer Advocates (DRA) and The Utility Return Network (TURN) support the ED recommendations with TURN specifically supporting a 15% allocation of EM&V funds to the IOUs.
SDG&E/SoCalGas advocate that the monetary split for EM&V projects between ED and the IOUs should be determined directly by the Commission, and should be determined and approved upfront in order to determine work load, study plans and appropriate staffing. SDG&E/SoCalGas would have the IOUs and ED each be responsible for their EM&V budgets and activities as set forth in D.05-01-055.
SCE asks that the Commission grant to the IOUs the same EM&V budgets approved for the 2006-2008 program cycle. Thus, SCE would modify the ED recommendation by: a) giving ED authority only for expenditures beyond the total EM&V budget granted to the IOUs for the 2006-2008 program cycle; b) requiring ED to consult with the IOUs before making EM&V expenditure decisions; and c) allowing the IOUs to use a dispute resolution process developed for EM&V.
PG&E also calls for the Commission to allocate a specific budget to the IOUs to conduct EM&V. PG&E notes that in the Joint Plan at 18, ED and the IOUs clarified that the Commission should allocate an EM&V budget to each IOU based on the IOU's proportional share of the total program budget. PG&E also notes that the Joint Plan at 18 estimated $49.5 million would be needed for process evaluation, market analysis and early EM&V. As IOUs are principally responsible for this category of EM&V, PG&E suggests that the allocation made to the IOUs should be equal to that sum.
There is no dispute that at least 15% (or $18.75 million) of the $125 million EM&V budget for 2010 through 2012 should be allocated to the IOUs to maintain staffing levels. This works out to approximately $1.5 million per utility per each of the three years. This appears to be a reasonable funding level for utility staffing. The question before us is whether the IOUs should be granted, as SCE puts it, discretion to use additional funds for particular categories of EM&V projects, specifically program design and market assessment studies, and early EM&V. This could result in up to $49.5 million of the $125 million total allocated to the IOUs by PG&E's estimate.3 Using SCE's methodology, the IOUs would be allocated the same $45 million allocated in D.05-11-011,4 including staffing costs. However, in D.05-11-011, this $45 million was 27.5% of the total EM&V budget of $163 million; the same amount would be 36% of the current $125 million budget for 2010 through 2012.
Our resolution of this issue relies on the overall context of this decision. There is agreement among parties that IOUs should conduct most or all of the program design and market assessment studies and early EM&V activities to assist in determining work load, study plans, and appropriate staffing. However, as we discuss below, the allocation and expenditure of these funds will be subject to limited ED review. In 2006-2008, the IOUs received 27.5% of the total EM&V budget. We will again allocate this proportion of the budget to the IOUs - that is, an additional 12.5% beyond the 15% agreed to in the Joint Plan to maintain IOU staffing levels. 27.5% of $125 million is $34.3 million; this will be the initial allocation to the IOUs. We require the IOUs to submit a report to ED within 15 days of the effective date of this Decision which documents the amount unspent as of the effective date of D.09-09-047.
4. The Joint Energy Division /Utility EM&V
Plan (Joint Plan)
The Joint Plan is included as Attachment 1 to this decision.5 ED and the IOUs agreed that we needed to develop a more collaborative and transparent working relationship as an important step towards improving the EM&V process. IOU and ED staff agree to work together on shared EM&V projects, and to follow mutually agreed upon standards for transparency, respect, and communication while working on separate EM&V projects. ED and the IOUs believe that this more collaborative process will result in greater cost‐efficiencies, more reliable results, broader stakeholder buy-in, and fewer disputed issues.
ED and the IOUs agree that the optimal approach to EM&V for 2010-2012 is to define an EM&V planning framework and retain flexibility to fund EM&V projects as needs arise, rather than to adopt from the outset a detailed plan covering the full scope of EM&V needs over a three year period. ED and the IOUs state that they will plan and implement EM&V projects with the goal of achieving the highest benefit for the EM&V expenditure, while minimizing interference with the programs and utility customers. To that end, EM&V projects will be designed to avoid duplication of effort, consolidated across all activities in a streamlined manner, and planned to comply with the Commission's schedule requirements.
The Joint Plan and its proposed budget allocations represent ED and the IOUs' current best judgment on the appropriate allocation of the authorized EM&V budget to EM&V projects needed to accomplish technically credible, quality work products that will comply with the Commission's requirements and goals stated in D.09‐09‐047. The Joint Plan presents a 2010 through 2012 EM&V Budget, with an estimated allocation of funding authorized by D.09-09-047 EM&V Projects. The estimated budget is shown on page 19 of the Joint Plan, also in Attachment 1 of this decision.
DRA generally supports the Joint Plan as long as ED's recommendations on other EM&V issues are incorporated into the EM&V framework (DRA's specific comments on issues other than the Joint Plan are discussed in sections below). DRA believes the Joint Plan needs improvement in the areas of transparency, and should include more opportunities for stakeholder input.
TURN supports generally the overall direction for EM&V set forth in the Joint Plan. TURN supports the Joint Plan's emphasis on a flexible and phased approach for the 2010-2012 portfolio period. TURN agrees with the Joint Plan's requirements that all IOU Phase 1 (immediately implementable) EM&V projects should be done in collaboration with ED to reduce the risk of such studies not being done in a timely manner.
There is no opposition to the Joint Plan. The Joint Plan presents a reasonable set of procedures to guide ED and the IOUs in formulating and carrying out EM&V studies. However, the Joint Plan does not address all outstanding issues. We will adopt the Joint Plan as presented in Attachment 1, subject to the discussion on remaining issues addressed herein.
In approving the EM&V plan, we wish to clarify that we intend for our staff to coordinate with other pertinent state agencies wherever such coordination enhances the state's overall energy policy goals. For example, we intend for the EM&V function to be supportive and responsive to the state's energy loading order policies and climate policy goals. We specifically direct our staff to coordinate with the CEC on the implementation of the energy consumptions surveys described in the EM&V plan as "EM&V Project Number 12," since the CEC has historically managed.
In the following sections we address additional issues which were not resolved as part of the Joint Plan. The determinations made here are informed by the ED Recommendations put forth in the November 20 ALJ ruling, and party comments that followed.6 We address these issues by topic, rather than by specific question as framed in the ruling.
5. Respective Roles and Responsibilities
of ED and IOUs
Many of the questions in the November 2009 ALJ ruling pertain to the division of EM&V roles and responsibilities between IOUs and ED, and the extent to which they should be modified. We have learned much since we addressed these topics in D.05-01-055. In concept, the division of EM&V responsibilities articulated in that decision remains sound. In practice, however, we have discovered a need for greater transparency and coordination than current processes have wrought. In some cases, ambiguity about roles and inadequate coordination have led to duplication of efforts at ratepayer expense. For example, SDG&E conducted a study using ratepayer funds to measure Compact Fluorescent Lamps (CFL) interactive effects, an activity beyond the scope of IOU EM&V responsibilities outlined in D.05-01-055, and which was ultimately duplicative of an on-going ED study.7 In addition, ED has found a number of other instances of duplicative studies performed by the IOUs with ratepayer funds, as well as studies arguably beyond the role of the IOUs set forth in D.05-01-055.8
As discussed below, we find that it is appropriate to make certain modifications to the roles set forth in D.05-01-055 and subsequently implemented for the 2006 - 2008 portfolio cycle, in order to minimize conflicts of interest, reduce duplication, and ensure transparency of information.
5.1. IOU-Managed Impact Evaluations (DEER)
Question 1.i of the November 20 Ruling asked: "Are the IOUs permitted to manage any impact evaluation or M&V projects that develop ex-ante savings estimates which may be used for determining portfolio performance, reporting accomplishments, or calculating incentives? If so, what are the Commission's expectations for rules and procedures for oversight of these projects?"
ED recommends that the IOUs should be permitted to manage projects to develop energy savings estimates9 in the specific case where there is no existing ex-ante estimate, or where the IOUs believe that an existing estimate is out of date and needs testing and ED is not already conducting or planning to conduct a project to develop estimates for the same measure. ED recommends that the IOUs be required to seek approval from ED before initiating such work and should proactively provide opportunities for ED to review project milestones and provide input directly to the project manager. ED seeks authority to oversee such projects, including authorization to deny approval of projects that are not in the ratepayers' interest.
PG&E agrees that it is generally the role of ED to conduct program and portfolio impacts-related studies, consistent with D.05-01-055. PG&E cautions against interpreting this provision to limit the IOUs' authority to conduct early EM&V studies. SDG&E/SoCalGas support the ED recommendations, with the caveat that the recommendations relate solely to studies that generate ex-ante savings estimates (as opposed to process evaluations and market assessments). Also, SDG&E/SoCalGas add that the approval process should be limited to no more than two weeks. SCE proposes specific language changes to ED's recommendations, which would have the effect of not giving ED sole authority to determine which studies may be done and when. For example, SCE would allow the IOU to proceed with a project if an ED project is scheduled to be completed more than three months later, as long as there is not duplication between projects.
We agree that IOUs should be allowed, under ED oversight, to manage projects to develop energy savings estimates in the specific case where:
1. There is no existing ex-ante estimate; or
2. An existing estimate is out of date and needs testing and ED is not already conducting or planning to conduct a project to develop estimates for the same measure.
Consistent with our policy to minimize conflicts of interest and conserve ratepayer funds, we will require the IOUs to seek approval from ED before initiating EM&V ex ante studies. ED shall have the authority to deny approval of projects. However, this authority is limited to situations where:
1. There is a conflict of interest with a contractor the IOU wishes to hire;10 or
2. There is duplication or significant overlap with studies already planned or carried out by ED; or
3. ED can specifically articulate why a study is unnecessary or inappropriately conducted by the IOUs.
We are sensitive to the need for timely oversight of ED. The proposal of SDG&E/SoCalGas that the approval process should be limited to no more than two weeks is reasonable and is adopted. Further, ED shall specify all decisions in writing, both to the IOU and posted on our website, and include its rationale for any denials.
Question 1.ii of the November 20 Ruling asked: "Is Energy Division expected and therefore permitted to initiate and manage evaluations that may be considered process or formative evaluations?"11
ED recommends the Commission authorize it to conduct any type of EM&V consistent with management of research projects that support the development of data, information, and tools needed to conduct regulatory oversight as well as to improve the Commission's energy efficiency policies. This may include the following types of research:
· Summative/ex-post impact evaluations.
· Evaluations and M&V conducted for the purpose of developing savings estimates.
· Evaluations and audits used to develop conclusions about program performance.
· Market studies required to inform Commission energy efficiency policies.
SCE recommends adding the following to the ED recommendation: "In cases where the IOUs are already conducting or planning to conduct process or formative evaluations on the same program or same topic, ED must coordinate with the IOUs to either conduct the study jointly under IOU management or to avoid duplication of data collection and attempt to make maximum use of the other's work in the later-starting study."
PG&E does not object to ED's request for authority to conduct market studies required to inform Commission energy efficiency policies, provided that such a request is not intended to restrict the IOU's ability to conduct market studies in accordance with their authority to do so under D.05-01-055.
We agree with parties that the fundamental purpose of process and formative evaluations is to inform program design and implementation, and that as program administrators it makes sense that the principal responsibility for managing such EM&V work lie with the IOUs. However, as the Commission takes a more involved role in program planning and improvement, and the development and tracking of program performance metrics as directed in D.09-09-047, we note that the ED should be permitted to manage evaluations that may be considered process or formative evaluations. This does not, however, imply a change from the original authority granted to ED EM&V work in D.05-01-055, which assigned to the ED management and contracting responsibilities for EM&V studies that will be used to "evaluate whether programs or portfolio goals are met."
Question 1.iii. of the November 20 Ruling asked: "Should ED have the authority to be involved in projects that develop ex-ante savings estimates, such as the non-DEER work papers, which are currently managed by the IOUs without any ED involvement?"
ED recommends that the IOUs should be required to notify ED of all workpaper12 development activities and should proactively provide opportunities for ED to review methodologies and provide input to the workpaper authors. ED contends that its involvement at this stage will streamline the review of final workpapers and will ensure greater reliability of workpaper savings estimates. ED recommends that its involvement in workpaper projects follow the process outlined in ED's recommendations for questions 4, 5 and 6 (see Attachment 3).
PG&E states that IOU workpapers regarding ex ante savings estimates are already subject to Commission oversight through the ED review and approval process, as set forth in an ALJ Ruling of November 18, 2009 in this docket. PG&E contends this level of review is sufficient and does not need to be enhanced as set forth in the ED recommendation.
The November 18, 2009 Ruling involved ED review of workpapers after submission to ED. We agree with PG&E that the process set forth in the November 18, 2009 ALJ Ruling is sufficient to provide Commission review of these workpapers after they are completed. ED seeks increased transparency in the initial development of the non-DEER workpapers. This is a valuable goal. We will require the IOUs to cooperate and collaborate with ED in the development of these workpapers.
Question 7 of the November 20 Ruling asked: How extensively should IOUs be involved in ED EM&V projects?
ED recommends adoption of its recommendations in Section C of the ED Straw Proposal "Stakeholder Input Process and Approval of EM&V Projects,"13 as well as the informal interactions proposed in the Joint IOU/ED EM&V Plan.
Overall, ED recommends that the Commission consolidate existing requirements for stakeholder input and restate those requirements in a comprehensive stakeholder input protocol for all ratepayer funded EM&V activities managed by either the IOUs or ED. The stakeholder input protocol would cover procedures for stakeholder and public review and input on EM&V project planning, development of savings estimates, publication of research findings, and the use of results produced by EM&V research projects. The stakeholder input protocol would include allowing time for stakeholder input in the overall EM&V project schedule, because, in the ED's view, the existing schedule and scope requirements do not allow sufficient time for interactions and information sharing.
SCE contends that only the portion of Section C of the ED Straw Proposal entitled "EM&V Project Implementation and On-Going Feedback" (at 8-9) is relevant to this question. This section of the Straw Proposal lays out the following process:
1. Energy Division and the IOUs will convene a meeting among their staff, EM&V contractors, stakeholders, and any interested member of the public to share key results and EM&V findings that might lead to improvements in the portfolio and identify best practices and possible improvements to evaluation methods. This meeting will take place sometime around the middle of the program cycle or at such time when significant results from various EM&V projects are available. If so requested by parties or members of the public, ED or IOUs, or both, should hold short informal meetings with groups or individual organizations, to discuss EM&V work progress and results.
2. ED and IOUs will convene ad hoc meetings (approximately quarterly) among ED staff, EM&V contractors, IOU EM&V staff and IOU program managers to discuss work progress and results. These meetings are to provide for timely feedback to program design and implementation. The IOUs can request meetings with ED to discuss work progress and results at any time.
3. When significant results are produced by the EM&V work, and a technical report is not immediately pending, the ED and/or the IOUs will provide informal written summaries of the results to the IOUs and other stakeholders. These written summaries will be posted on the same website used for posting EM&V work plans and comments.
PG&E and SDG&E/SoCalGas recommend retaining the process set forth in D.05-01-055 at 115 - 118. PG&E cites the Joint Plan's agreement that informal ED/IOU interactions based on general principles should not "impose formal or specific obligations on the ED or the IOUs and do not define the formal division of EM&V roles and responsibilities." PG&E thus argues that the level of participation by IOUs in ED projects should not be spelled out as proposed in the straw proposal by ED.
SCE is correct that only one portion of Section C of the Straw Proposal is relevant to this question. ED's recommendations in this portion of Section C are reasonable, and should be adopted. While these recommendations provide a certain amount of detail, they are not onerous or burdensome. The recommendations provide significant leeway in timing and detail necessary to carry out these responsibilities.
D.05-01-055 did not formally adopt a process for IOU involvement in ED EM&V projects. D.05-01-055 did hold that ED should involve program implementers (which mostly consist of IOUs) in technical discussions concerning ED's projects. ED's recommendations are intended to accomplish the goals envisioned in D.05-01-055: to ensure that stakeholders have access to EM&V results in an orderly and timely manner, in order to improve energy efficiency portfolios. We adopt the ED recommendations regarding stakeholder input, which modify and supersede the process adopted in D.05-01-055.
Questions 2, 3, and 4 of the November 20 Ruling asked: "Should ED be responsible for approving IOU EM&V projects? Should there be exceptions to this process for expedited projects?"; "Current policy requires ED to approve all IOU EM&V contractors in order to manage contractor conflicts of interest. Should this process continue or be modified?"; and "Should ED have the authority to be involved in IOU EM&V projects?"
ED recommends that ED's involvement in authorizing and reviewing IOU EM&V projects, including ex-ante savings estimation projects, should be managed according to procedures adapted from the ED Straw Proposal. The ED Straw Proposal on this point is included as Attachment 2 (as modified by the discussion below). ED also suggests that it would exercise the authority granted to Commission staff under Public Utilities Code § 314 (a),14 as needed, to review process evaluation plans and results.
PG&E argues that ED should not be responsible for approving IOU EM&V program design and market assessment studies, for which explicit management authority was delegated to the IOUs in D.05-01-055. PG&E claims the IOUs cannot be expected to meet program goals if they do not retain sufficient authority to evaluate their ongoing programs as they deem necessary. Regarding management of contracts, PG&E requests that the Commission clearly state which criteria are appropriate to support a finding that a conflict of interest exists, and which are severe enough to support rejection of a proposed contractor. PG&E proposes creation of a pre-approved contractor list to minimize conflicts.
SDG&E/SoCalGas generally agree with PG&E. SDG&E/SoCalGas recommend that if ED's recommendation is nevertheless adopted, the time frame referenced in Section 4.2 above (notification to the IOU if ED intends to hold the project) should be decreased from two weeks to one week.
SCE is willing to accept ED's recommendations with three provisos. First, SCE would change ED's authority regarding studies from "review and approve" to simply "review." Second, SCE wishes to reserve the right to use a dispute resolution process if there are repeated instances of studies being substantially delayed without good reason. Third, SCE would clarify that the reason for overturning an IOU's selection of a preferred contractor must be due to ED's identification of a meaningful conflict of interest for the proposed contractor, with a description of the alleged conflict of interest, before a final contractor is selected. This would allow a full discussion of the conflict issues between the IOU and ED, so that any problem could potentially be resolved at that point.
SCE and SDG&E/SoCalGas also state that some project reporting documentation may be confidential and should not automatically be made public.
We will accept the ED recommendations, with modifications. As is current practice, we agree that material properly designated to be confidential should be kept confidential by ED. Consistent with the review process adopted for ex ante studies above, we will streamline the process to adopt a review period of two weeks for ED determine if ED needs to hold approval of a project. We agree with SCE that ED should provide the IOU with a written statement explaining the specific conflict problem behind rejection of a proposed contractor, and should provide an opportunity for discussion or conflict resolution before a final contractor decision is made. However, consistent with streamlining the process, the IOU shall have two weeks from the date of receiving the written statement of contractor rejection to discuss the issue, or file a "Motion for EM&V Dispute Resolution" as adopted herein. Otherwise, ED may finalize the selection of contractor.
Attachment 2 sets forth the process we adopt today.
SDG&E/SoCalGas recommend that IOU process evaluations and market assessments should not require approval from ED, but agree that ED could have input to a list of possible contractors for the studies.
As we have discussed above, we are altering the delineation of roles as spelled out in D.05-01-055 to minimize conflicts, avoid duplication and improve transparency. Placing ED in the role of reviewing what EM&V studies should be conducted by the IOUs and what studies should be conducted by ED fits squarely within our policy.
However, we are not making a determination here that IOUs should not manage or conduct certain studies, or that ED should manage or conduct certain studies; in other words, we do not determine that there necessarily should be any change in who manages or conducts process or formative evaluations. There are good reasons why the IOUs have taken the lead in these areas in the past; the IOUs may conduct such studies in a more timely manner, and can use the results to help improve development of or implementation of energy efficiency measures. As with IOU EM&V studies, if ED rejects an IOU proposal for a study, it should reject the request by providing the IOU, within two weeks of the IOU request, with a written statement indicating rejection due to duplication, lack of necessity or conflict of interest.
Question 8 of the November 20 Ruling asked: "What is the appropriate level of public involvement in EM&V projects? Should certain EM&V project be exempted from a full public process? How will the exempted EM&V projects be determined?"
ED recommends a comprehensive stakeholder input protocol for all ratepayer funded EM&V activities managed by either the IOUs or ED. ED's recommendations are laid out on pages 8-11 of the Straw Proposal.
In their comments on the ED Straw Proposal, the IOUs expressed concerns that engaging with the public on every EM&V project would be ineffective and would slow down the implementation of time-sensitive projects. ED believes that there will be IOU EM&V projects that will not require an intensive public vetting process, but does not believe the project budget is a reasonable indicator of the need for public vetting. Additionally, ED believes that ratepayers and the Commission are key stakeholders for process evaluations. To ensure that the appropriate EM&V projects are publically vetted and that time-sensitive projects are not delayed, ED recommends that the Commission grant ED authority to determine which EM&V projects should and should not undergo public vetting.
DRA recommends an annual public overview of program design and implementation for stakeholders, including a feedback opportunity. PG&E asks the Commission to clarify the criteria that would be used to determine what public input should be required on EM&V projects, as opposed to allowing ED to make a case-by-case determination. SCE would accept the ED recommendation, except that SCE would allow disputes on this topic to be resolved via the dispute resolution process adopted herein. TURN would also allow the dispute resolution to be used to challenge whether a certain subject should be exempted from further public vetting. SDG&E/SoCalGas agree with the ED recommendation, except that they argue the size of the project should be a good criterion for determining the need for public vetting.
While we agree in principle with PG&E that specific criteria should be established to determine the level of public vetting for EM&V projects, it is very difficult (and there is not a sufficient record) to determine such criteria. Parties are correct to point out that project size is not a good proxy for need for public input, but no other specific criterion has been put forth. In general, public involvement should be sought to the maximum degree feasible, yet the cost and time involved may make such effort unproductive in some cases (such as small or short timeframe projects). Thus, it makes sense to delegate to ED the responsibility to make such determinations. Consistent with our policy that timeliness should be taken into consideration, ED should weigh the value of public input versus the extra time such input would entail. With this caveat, we will adopt the language in the ED recommendation from the Straw Proposal with the addition that a party may file an EM&V Motion as described herein.
Question 9 of the November 20 Ruling asked: "Should all IOU EM&V related projects, regardless of funding source (such as projects that develop savings estimates for non-DEER measures funded out of program funds), be required to follow the same policies and procedures that are required for EM&V funded projects?"
ED recommends that the Commission require that all EM&V-related projects, regardless of funding source, adhere to the same policies and procedures as EM&V funded projects.
TURN agrees with the ED proposal, arguing that this will counter any incentive that exists for IOUs to make small changes to a DEER measure and present it as a new non-DEER measure which may receive less scrutiny. DRA and SDG&E/SoCalGas agree with the ED recommendation. PG&E agrees that all EM&V projects should be funded from the EM&V budget, not the program budget. PG&E contends that IOU research projects which are properly designated as program implementation activities (and thus paid for with program implementation dollars) should not be subject to rules and procedures designed for EM&V. SCE similarly argues the ED language is too broad, claiming that the term "EM&V-related" could be attached to many projects not usually considered as EM&V projects.
We will adopt the ED recommendation, with the caveat that the EM&V processes adopted herein should not apply to projects not previously considered to be in the EM&V category. For example, non-DEER studies would be considered EM&V projects, while (as SCE suggests) developing initial workpapers using existing data sources would not be considered as EM&V.
6. Dispute Resolution Mechanism
Question 6 of the November 20 Ruling asked: "How should major disputes arising out of the EM&V work be managed? When should these disputes be elevated to the full Commission for resolution?"
In the Joint Plan at 8, ED and the IOUs state: "(i)t may not always be possible or productive to reach consensus between ED and IOU staff during the planning and implementation of EM&V projects or interpretation and use of EM&V results. ED and the IOUs will seek to achieve consensus through informal processes. If consensus cannot be reached informally, ED and the IOUs will follow the applicable dispute resolution processes in effect wherever a formal resolution is necessary."
ED has recommended the following in the ED Straw Proposal:
· For Project-Specific EM&V Plans, if parties continue to take issue with the final work plans, a party or parties may file a motion with the assigned ALJ and provide a rationale for why the plans should be changed and how. The ALJ will resolve the dispute and direct ED and/or the IOUs to revise the plans accordingly via ruling.
· For EM&V Technical Reports, if parties continue to take issue with the final EM&V technical reports, a party or parties may file a motion with the assigned ALJ and provide a rationale for why the report is deficient and what changes to the report would be necessary to correct the deficiency. The ALJ will resolve the dispute and direct ED and/or the IOUs, via ruling, to prepare an addendum to the report correcting the deficiency. The addendum will be posted on the same website where the draft reports are posted.
DRA and TURN support the ED's recommendations. NRDC recommends formulation and implementation of one or more formal dispute resolution mechanisms specifically tailored to EM&V.
SCE advocates that dispute resolution should be attempted through informal processes before being elevated to ALJ or Commission resolution, and that major disputes be resolved through Commission decisions. PG&E proposes initial submission of disputes, which are often technical in nature, to an independent, expert evaluation body for resolution. Unlike ED, PG&E would allow for escalation of disputes concerning project-specific plans and review of technical reports to the full Commission, not just the assigned ALJ. SDG&E/SoCalGas propose that major disputes should lead to an ALJ PD, with final determination by the full Commission.
Currently, disputes between the IOUs and ED regarding EM&V studies are resolved by ED, with no specific process for appeal. This is consistent with the relationship that ED has with the IOUs: ED as an arm of the regulatory body is carrying out the Commission's policy through delegated authority. IOUs have been frustrated at times with this model. Without a formal appeal process, IOUs would need to use another mechanism to challenge ED determinations, such as was done in a filing a Petition for Modification of D.07-09-043 and D.08-01-042 in 2008 in effect to appeal ED's first Verification Report of 2006 through 2007 energy savings.15
As PG&E points out, the process surrounding contractor selection and determination of EM&V study topics involves many technical issues. Certainly, the determination of energy savings involves a variety of technical assumptions and calculations, with a high potential for differing opinions. It is reasonable for certain disputes regarding complex and controversial EM&V matters to be resolved by ALJs and/or Commissioners instead of by ED staff.
All parties, as well as ED, now agree there is a need for a new dispute resolution process with regard to EM&V studies. The first priority should be to minimize any formal disputes. The best way to do so is to ensure open and full communications between ED and IOUs, as well as transparency for the public. Avoiding misunderstandings, developing trust, and providing transparency should go a long way toward avoiding or resolving potential issues before there is a need to escalate to a formal dispute resolution process. The rules for ED review adopted herein and set forth in Attachment 2 provide an orderly process which should help minimize disputes upfront.
ED is not a formal party to our proceedings. This means that ED does not present witnesses, file formal comments, present sworn testimony or have other rights or obligations of parties. Yet, at this time, for EM&V ED serves in a dual role of independent evaluator and (in the case of a formal dispute) advocate to Commission decision-makers for its analysis and decisions.
We do not wish to confer party status on ED for these purposes. To do so would be to compromise the ability of ED to perform its essential function of impartially and confidentially advising the Commission. It would be impractical to set up an "advocacy" portion of ED to engage in EM&V dispute resolution, apart from the overall "advisory" portion of ED, if for no other reason than the same personnel would have to wear two hats (or additional staff would be required).
To find our way through this issue, we look to previous efforts in the energy efficiency area. In recent years, ED proposals have at times been issued for comment by ALJ and/or assigned Commissioner Ruling. This has provided parties a formal opportunity to comment on such proposals.16 As with a Motion or Petition for Modification, ED does not have an automatic right to reply to comments on the Ruling.17 However, there has been no prohibition on ED advising the ALJ, assigned Commissioner or other decision-makers on these matters.
We do not agree with ED that the assigned ALJ should be the final arbiter of all formal disputes. This approach would put an undue burden on the ALJ and would place too much delegated authority to one individual. Instead, we will provide multiple avenues for dispute resolution.
We will provide for the following dispute resolution methods for those matters which cannot be resolved informally. A party may file a "Motion for EM&V Dispute Resolution" (EM&V Motion) with the assigned ALJ in R.09-11-014 or its successor for resolution of an EM&V matter. The EM&V Motion must include a statement from ED giving its side of the dispute.18 The ALJ may undertake any appropriate process to gather further information. The ALJ may issue a Ruling to resolve the dispute.
Alternatively, in an EM&V Motion, the filing party or the ED may ask that the matter be resolved by the assigned Commission or the full Commission. In that case, the ALJ will consult with the assigned Commissioner to determine the appropriate course of action. This may include an ALJ Ruling, an assigned Commissioner Ruling, or a Commission Decision. If the ALJ and assigned Commission decide to bring the matter to the full Commission, the ALJ or assigned Commissioner will issue a PD and allow for comment under Rule 14 of the Commission's Rules of Practice and Procedure.
The EM&V Motion may be used for the following purposes only:
· Disputes over results of EM&V studies or reports;
· Dispute over selection of an EM&V contractor;
· Disputes about project-specific final EM&V work plans;
· Disputes regarding final EM&V technical reports; and
· Disputes concerning public vetting of EM&V projects.
The EM&V Motion process does not apply to any dispute over results of ED Verification Reports, either draft or final.
SCE, NRDC, and SDG&E/SoCalGas in comments to the PD recommend that EM&V process and/or EM&V disputes should include review by non-Commission experts. We do not adopt this proposal. ED and its EM&V consultants are inherently independent of any parties' interests. Review of ED EM&V work by non-Commission experts would be redundant, costly, time-consuming, and would only serve to undermine public confidence in ED's efforts. The new dispute resolution process will be a better way to address any technical concerns.
7. Customer Participation in EM&V as a
Condition of EM&V Funding
Question 10 of the November 20 Ruling asked: "Should the IOUs modify program eligibility rules to require very large customized program participants to participate in evaluations if selected in a sample, as a condition for receiving energy efficiency funding?"
ED has found that many large project participants have refused to participate in evaluations. ED believes it is reasonable to require participants who receive a large sum of energy efficiency funding and services to participate in evaluations, if needed. This participation would include on-site measurement and verification, as well as surveys of key participant personnel. ED proposes to review past projects with the IOUs to determine the energy efficiency incentive threshold above which participation in evaluations would be obligatory.
ED pledges to reduce the burden of participating in evaluations by coordinating with the IOU implementation and inspection process. ED recommends that the Commission require the IOUs to cooperate with ED in this regard.
SCE sees this as a program issue, not an EM&V issue. SCE asserts that the IOUs already require such customers who receive energy efficiency program funds to sign a statement acknowledging that they may be required to participate in evaluation studies. If this issue needs to be considered further, SCE would defer this issue to R.09-11-014. PG&E and SDG&E/SoCalGas agree with SCE that large customized program participants are already subject to evaluation requirements, but PG&E notes that this obligation is limited to calculated savings programs (as opposed to deemed savings programs).19 SDG&E/SoCalGas recommend that ED and the IOUs improve coordination in this area to ensure better customer participation in the evaluation process. DRA agrees that streamlining the EM&V process and accommodating customer needs is important.
This issue appears to be one of enforcement of existing obligations. While all customers receiving energy efficiency program funds are obligated to submit to evaluation studies, some customers apparently refuse to do so. This may be because, as SDG&E/SoCalGas point out, customers may find the studies to be redundant, to require too much effort, or to be intrusive. Nevertheless, we emphasize to both IOUs and customers that our energy efficiency program relies on accurate information about programs and we expect all participants to adhere to evaluation requirements.
We find no specific policy that needs to be changed here; the current requirements simply need to be enforced. We encourage the efforts of ED (including its EM&V consultants) and the IOUs to work with customers to both ensure necessary and required cooperation and to limit the burden on customers.
8. Counting Savings from
Behavior Based Programs
In D.09-09-047 we directed an investigation into the feasibility of crediting savings from behavior-based energy efficiency programs. In following with this directive, Question 10 of the November 20 Ruling asked: "Should the Commission allow the IOUs the opportunity to count savings from behavior based programs? How should the Commission develop EM&V methodologies to verify savings driven by behavior-based efficiency programs? What analytical issues are raised by changing policy to allow credit and require measurement of savings driven by behavior-based efficiency programs (i.e. savings persistence, potential double-counting of savings by other resource programs, potential double-counting of savings claimed as part of the conservation benefits assumed to underlie Advanced Meter Infrastructure (AMI) business cases [PG&E D.09-03-026; SCE D.08-09-039; SDG&E D.07-04-043])?"
ED believes that the categories of behavior-based programs must be well-defined and measurement issues clarified before categorically recommending savings credit from behavior based programs, stating that if defined too broadly, evaluation resources could become unduly tied up in measuring savings from which represent a small fraction of overall portfolio savings. In addition, ED believes there are significant intersecting issues with the IOUs' AMI programs. For instance, ED believes it is the intent of the AMI program to provide customers with usage data to help them manage their energy consumption through conservation. Comparative usage reporting and benchmarking could be provided as part of the bundle of AMI services. Thus, crediting these savings in the context of energy efficiency programs will require careful accounting to ensure that they have not been either counted, or paid for, twice. ED recommends that the Commission consider forming a working group, facilitated by ED, to explore these issues.
For the purpose of this Decision, we will restrict the definition of behavior based programs to the "comparative energy usage disclosure programs" defined in SB 488. As defined by SB 488, comparative usage programs are specifically programs "...pursuant to which an electrical corporation or gas corporation discloses information to residential subscribers relative to the amount of energy used by the metered residence compared to similar residences in the subscriber's geographical area." We understand that certain non-residential market sectors may offer opportunities for comparable or greater savings under similar programs, and thus do not restrict our definition to residential applications.
PG&E points out that Senate Bill (SB) 48820 encourages the pursuit of comparative home energy reporting, a specific type of behavior based program, which should be evaluated using experimental design approaches. PG&E and OPower suggest that the use of experimental design can help prevent double-counting from behavior based programs and other initiatives, such as AMI. OPower provides two examples of completed experimental studies of their program as well as a proposed protocol, which PG&E and Sempra recommend be added to our evaluation protocols. PG&E, SCE, and OPower point out that experimental design is already a method accepted by the Commission and is included in the California Evaluation Protocols.21
DRA recommends that ED develop a methodology to isolate behavior based savings from EE resource programs, non-EE programs, AMI influences, as well as energy efficiency messages from sources outside of the utilities. DRA is concerned that measurement of behavior based savings without improved methods will be daunting, costly, and controversial; and questions whether such methods can be implemented objectively. TURN believes that extensive research is still needed before the Commission should commit to allowing the counting of savings from behavior based programs. PG&E believes that the Commission must act quickly with existing methods and not wait to develop any new methods or protocols.
SCE requests that the Commission to create a regulatory environment that encourages behavior based programs, emphasizing that behavior change and conservation are critical to achieving market transformation and reductions in energy use. SCE recommends a performance incentive mechanism that tracks and credits behavior and conservation goals, including the measurement of energy savings from behavior based programs. OPower states that the IOUs have indicated that they want to operate behavior-based efficiency programs at scale, and that they are waiting for this Commission's approval before they do, and in at least in one case, will only proceed with such a plan if the Commission recognizes behavior as an efficiency resource. SCE argues that the greatest uncertainty with measuring behavior based programs is savings persistence. SCE therefore suggests that savings from behavior based programs be estimated with shorter ex-ante effective useful lives while ED and the utilities identify and develop more reliable methods for estimating energy savings created by programs focused on changing energy user behavior. NRDC suggests that behavior based programs be assumed to provide savings only during the period that the program is in place.
SDG&E/SoCalGas supports the estimation of savings from behavior based programs, so long as the savings are reliable and verifiable. SDG&E/SoCalGas advises that the full value of benefits resulting from California's Smart Meter (also known as advanced metering infrastructure or AMI) investment must be recognized and quantified to encourage effective utility programs.
In D.09-09-043, we indicated our intent to consider EM&V methodologies to account for behavior based programs. We reasoned that because many of the programmatic directives in that decision marked a shift towards market transformation consistent with Strategic Plan objectives, the Commission should consider ways to credit savings from new programmatic approaches focused on generating measured energy use reductions through behavioral motivation.
We agree with the utilities that it is within our energy efficiency program's best interest to create a regulatory environment that encourages behavior change and conservation. Such a regulatory environment necessarily includes the measurement and crediting of energy savings from programs that focus on behavioral motivations to generate measurable savings. We also understand ED's concern that if defined too broadly, this pursuit could consume many resources and distract away from other priority EM&V activities. As DRA warns, we must be cautious not to commit to an overly complex, costly, and controversial measurement system. All parties recognize the need to avoid double-counting, with which we agree.
Given the parties' comments and testimony submitted on this issue, we are persuaded that it is reasonable to measure savings from certain behavior based programs. To the extent that any program holds potential to produce significant verifiable savings, it is appropriate to attempt to estimate such savings, particularly when consistent with the overall policy direction we have adopted for energy efficiency.
We thus adopt a policy to measure and count savings from comparative usage programs, as defined in this Decision, using experimental design methodologies contained within the California Evaluation Protocols. We defer to the prioritization process described in the EM&V Plan adopted in this decision to make decisions regarding which behavior based programs will be evaluated and specifically how those programs will be evaluated.
These programs have intersections with several other categories of program activities already underway, such as AMI. As such we must take special care to ensure that savings credited to these programs do not represent double-counting. The experimental design method, as described in the California Evaluation Protocols, and spelled out in greater detail in parties' comments, is well equipped to deal with most of the analytical issues raised by the overlap of the savings targeted by comparative energy use reports, and programs already under way through Commission directive. So long as the evaluation is set up to compare two populations which in statistical terms are in no way different except for the treatment of the program, the measured savings should be those attributable to the program, provided the experiment is properly designed.
As shown in the record, deployments of comparative energy use programs have yielded savings on the order of 1.5 to 3.5 percent across sample populations. However, due to overlap with other initiatives targeting behavior change, measurable savings from service areas within our jurisdiction may differ, lending uncertainty to any savings we may project for these programs. We thus, propose a slightly different process for the crediting of savings from these programs for 2010-2012. Utilities will not be allowed to submit workpapers for ex ante numbers which draw from other sources to project savings for these programs. Our policy determination in D.09-09-047 to freeze ex ante values could potentially lock in overly optimistic savings projections for this novel brand of resource program. Instead we commit only to crediting ex post savings for behavior programs in the 2010-2012 program cycle. As such, the onus is on the program provider to make the case to the utility that the program provides added value to efforts already underway, and that projected savings will materialize as real and verifiable.
9. Other Issues Deferred by D.09-09-047
9.1. EM&V Contractor Firewall Issues
In D.05-01-055 at 122 we established a policy that prohibited firms engaged in energy efficiency work from performing both program impact evaluations and program implementation:
"Specifically, we will prohibit entities from performing any program and portfolio impacts-related studies at the same time they are under contract for program delivery work. As defined in this decision, these are the types of studies that are designed to produce findings (that may be favorable or unfavorable) on program or portfolio accomplishments."
In the Straw Proposal, ED proposed making case by case exceptions to this firewall policy in order to recruit program implementers to collect data needed for EM&V, following strict protocols. All parties except SDG&E/SoCalGas support ED's proposed modifications, while SDG&E/SoCalGas opposed any changes to the current regimen. ED's approach is reasonable because it will allow a more efficient use of resources in certain situations where otherwise multiple implementers or evaluators would be needed to conduct site measurements. We will modify the firewall to allow ED, on a case by case basis, to use program implementers as a vehicle for collecting EM&V data when this would clearly be more efficient.
In the Straw Proposal, ED proposed changes to the avoided costs adopted in D.06-06-063 for the purpose of calculating benefits from the 2009 bridge funding period. Specifically, ED proposed the development of a new set of electric avoided costs using the Combined Cycle Gas Turbine (CCGT) costs from the Market Price Referent (MPR), including transmission and distribution costs and an update to the gas price forecast based upon the following options:
1. the 2006-2008 interim avoided costs from D.06-06-063 and an escalation factor for years not covered in the interim values; or
2. current market values (consistent with the approach authorized in D.06-06-063); or
3. using the market values obtained for the 2009-2011 planning values.
In comments to the proposed decision, SCE suggests a fourth option using the gas prices and capacity values that are contained in the 2008 MPR specifically for 2009 programs. SCE further recommends that ED use current gas prices (option 2) for purposes of updating the 2008 MPR's natural gas prices for evaluation of the 2010-2012 programs. We direct ED to update the avoided costs using the best of the proposed options at the time of the update.
ED also recommends that the greenhouse gas (GHG) adder be updated using the 2008 MPR value of $30 per tonne. No party opposed ED's proposal to update the avoided costs for 2009. SCE expressed concern that the GHG adder levelized cost calculation over-values short lived measures, but provided no further detail or examples. ED should ensure, to the extent possible, that the carbon adder calculation methodology does not over-value short-lived measures.
D.06-06-063 is modified to adopt a new set of electric and gas avoided costs to be applied to energy efficiency from 2009 forward. The electric and gas avoided cost should include a modified CO2 emissions adder of $30/tonne in 2009 using generation cost inputs from the most recently adopted Market Price Referent as of the date of this Decision and updating the natural gas cost forecast using data as of the date of this Decision.
In the Straw Proposal, ED proposed using results from the final 2006-2008 evaluation reports as inputs for calculating the energy impacts of 2009 programs, for those measures and programs that were evaluated during the 2006-2008 period and also extended during 2009. PG&E believes that DEER 2008 values rather than the 2006-2008 ex-post values should be used as inputs for calculating the impacts of the 2009 programs. PG&E argues that the DEER 2008 values are the most appropriate inputs for calculating the energy impacts because these were the values available to the IOUs when planning their 2009-2011 programs. SCE agrees, but recommends that ED extend the 2006-2008 evaluation contracts to gather representative samples for the 2009 programs. TURN and DRA agree with ED's proposal. TURN, DRA and PG&E disagree with SCE's proposal to conduct additional field work on 2009 programs for the purpose of calculating the energy impacts of 2009 programs.
We adopt ED's proposed approach to calculating the energy impacts of 2009. While PG&E cites language in D.08-10-027 that directed the use of 2008 DEER in calculating the savings and cost-effectiveness of 2009 programs, that directive was put into place to address discrepancies between IOU filings and certain DEER values, and not to limit the applicability of 2006-2008 evaluation results to 2009 programs. The determination to "freeze" cost and savings assumptions in D.09-09-047, cited by SCE, was made well after D.08-10-027, and was meant to apply on a forward looking basis. We therefore reject proposals not to evaluate 2009 programs using 2006-2008 EM&V studies. The energy impacts of the 2009 programs shall be reported by ED before the end of 2010, or as otherwise required by R.09-11-014, R.09-01-019, or other applicable energy efficiency docket.
D.08-10-027, which authorized the 2009 bridge funding, did not make clear how much of the authorized funding would be allocated between ED and IOU managed projects. We find that it is appropriate to use the same allocation between ED and IOUs for the 2009 EM&V funding as was used in the decision approving 2006-2008 EM&V budgets (D.05-11-011). The allocation rate adopted in that decision is 27.5% to the IOUs and 72.5% to ED.22
In D.09-09-047 at 205, the Commission deferred four energy efficiency policy rule modifications related to measurement of the codes and standards program (C&S) proposed by the IOUs in their second amended application for approval of 2010-2012 portfolios:
In their July, 2009 Second Amended Application, the utilities propose to: 1) count 100 percent of gross savings from all proceedings including pre-2006 advocacy efforts towards minimum performance standard and performance earnings basis; 2) gain credit for savings achieved through the Compliance Enhancement and Reach Code sub-programs, 3) clarify the calculation methodology of gross savings for C&S; and 4) reconsider and calculate savings resulting from non-utility territories. These issues will be deferred to the forthcoming decision in this docket on EM&V issues.
D.09-09-047 provides a summary of the parties' comments on the IOUs second amended applications and in response to questions posed in the June 9, 2009 ALJ Ruling issued in Application (A.) 08-07-021.
We make changes to the way we measure codes and standards for the 2010-2012 program cycle. In D.07-10-032, it was ordered that the current Commission policy will continue for the 2009-2011 (now 2010-2012) program cycle; i.e., we will count 50% of verified savings from the IOUs pre-2006 C&S advocacy work towards achievement of goals for 2010-2012, and 100% of verified savings from post 2006-08 C&S advocacy work. We are convinced by the IOUs and NRDC that 100% of verified savings from pre-2006 C&S advocacy work should count toward achievement of 2010-2012 goals. As PG&E points out, better technical data about savings is now available as compared to when the original 50% determination was made in D.05-09-043, including Evaluation Protocols and elimination of concerns about double-counting and base case forecasts. We clarify that this accounting is only for savings occurring within the IOU service areas.
We also clarify that gross savings should be calculated using the unit energy savings for each standard or measure adjusted for installation rates and non-compliance. The baseline for gross savings should be the previous standard or the prevailing market practice.
Net savings should be the gross savings adjusted by the rate of "naturally occurring market adoption" (NOMAD) and the attribution level for each IOU within the IOUs service territories.
It is important to clarify what is involved in the concept of a "Reach Code." By their nature, the code must be formally adopted by an enforcement jurisdiction. The code must be legally enforceable and enforced by the jurisdiction, and it must apply to all entities within the adopting jurisdiction. It may cover extensions beyond current Title -24 and Title 20 standards, or it may involve new technologies or practices. By the nature of Reach codes, all measures adopted wouldn't necessarily have to be currently cost-effective. We direct ED staff to conduct pilot evaluations of the Compliance Enhancement and Reach Code sub-programs.
10. Comments on Proposed Decision
The proposed decision of the ALJ in this matter was mailed to the parties in accordance with Section 311 of the Public Utilities Code and comments were allowed under Rule 14.3 of the Commission's Rules of Practice and Procedure. Comments were filed on March 29, 2010, and reply comments were filed on April 5, 2010.
Assignment of Proceeding
Dian M. Grueneich is the assigned Commissioner and David M. Gamson is the assigned Administrative Law Judge in this proceeding.
1. EM&V is an important part of the Commission's energy efficiency program for several reasons. First, it is necessary to determine whether and how well current individual programs are working, both in terms of saving energy and in comparison to projections. Second, EM&V is critical in considering how to improve programs and for development of new measures. Third, EM&V is used on a broad level to measure whether the IOUs are meeting, on a portfolio basis, the overall energy savings goals established by the Commission. Fourth, EM&V results are used to determine whether IOUs should receive rewards or pay penalties as part of the energy efficiency shareholder incentives plan developed by the Commission.
2. D.05-01-055 and D.05-11-011 set forth the rules for EM&V for the 2006 through 2008 energy efficiency program.
3. D.09-09-047 adopted energy efficiency portfolios for 2010 through 2012 for SCE, SoCalGas, SDG&E, and PG&E. That decision established the Commission's goals and core objectives for EM&V for the 2010 through 2012 portfolios.
4. On July 7, 2009, an ALJ Ruling sought comment on an ED "straw proposal" on EM&V issues. In the July 7 Ruling, parties in this proceeding were given notice that the Commission may in this proceeding adopt changes that would modify D.05-01-055. To that end, the July 7 Ruling with its attachments was served on the service list in R.01-08-028 (the proceeding in which D.05-01-055 was issued).
5. A budget of $125 million, or 4% of the overall portfolio budgets, for 2010 through 2012 EM&V was adopted in D.09-09-047, subject to review in this decision.
6. In previous energy efficiency cycles, each utility's EM&V budget was determined by its proportional share of the total EM&V budget approved by the Commission, with the proportion equal to its proportion of total program budgets. For the 2010 through 2012 energy efficiency cycle, these proportional amounts are: 43% for PG&E; 39% for SCE; and 9% each for SDG&E and SoCalGas.
7. ED and the IOUs prepared an EM&V plan which was jointly submitted to the assigned ALJ and issued for comment via Ruling. The Joint Plan is responsive to the Commission's stated desire in D.09-09-047 to make near-term improvements in order to streamline EM&V processes, and enhance timeliness, transparency and consistency across EM&V work products and to take a fresh look at several aspects of our EM&V activity in California for the upcoming program cycle, to reduce unnecessary burden on staff and other resources, and streamline our EM&V processes.
8. Both ED and IOUs should have specific and defined roles in EM&V.
9. IOUs continue to have a vested interest in the outcome of EM&V studies, as these studies are used to determine the level of energy efficiency shareholder incentives.
10. As program administrators and implementers, the IOUs have the data and the expertise to provide critical input to EM&V studies. IOUs also have a role, along with other energy efficiency constituents, to use the data gathered in EM&V efforts to improve program development and implementation.
11. There is evidence of overlap between IOU and ED EM&V activities that reduces the effective use of ratepayer funds.
12. EM&V studies should help form the basis for improvement of energy efficiency programs by showing what works well and what does not. It is important that this information be made available to stakeholders to the greatest degree feasible without compromising confidential information.
13. The IOUs require 15% (or $18.75 million) of the $125 million EM&V budget for 2010 through 2012 to maintain staffing levels.
14. $34.3 million is a reasonable estimate for the funding needed for IOUs to perform EM&V studies in 2010 through 2012, including the amount needed to maintain staffing levels.
15. IOUs have in the past conducted some or all of the program design and market assessment studies and early EM&V activities.
16. The determination of energy savings involves a variety of technical assumptions and calculations, with a high potential for differing opinions.
17. Currently, disputes between the IOUs and ED regarding EM&V studies are resolved by ED, with no specific process for appeal. All parties agree that increasing contentiousness in this area requires the Commission to create a new EM&V dispute resolution process.
18. ED is not a party to Commission proceedings. If an EM&V dispute is made formal through a Motion or other action by a party, ED would not have a right to file comments disputing the party's version of the dispute.
19. ED provided recommendations regarding IOU involvement in ED EM&V studies in Section C of the ED Straw Proposal "Stakeholder Input Process and Approval of EM&V Projects," which are complemented by the informal interactions proposed in the Joint Plan.
20. ED provided recommendations regarding determining the level of public vetting for EM&V projects.
21. ED recommends that the Commission require that all EM&V-related projects, regardless of funding source, adhere to the same policies and procedures as EM&V funded projects.
22. ED recommends that it review past projects with the IOUs to determine the energy efficiency incentive threshold above which customer participation in evaluations would be obligatory.
23. D.05-01-055 established a policy that prohibited firms engaged in energy efficiency work from performing both program impact evaluations and program implementation. This policy at times limits the ability to recruit program implementers to collect data needed for EM&V.
24. Avoided costs adopted in D.06-06-063 need to be updated for the purpose of calculating benefits from the 2009 bridge funding period, including an updated GHG adder, and for 2010 and forward.
25. There are significant intersecting issues with the IOUs' AMI programs and energy efficiency behavior-based programs, which could lead to double-counting of benefits.
26. It is reasonable to attempt to measure savings from certain behavior based programs.
27. The experimental design method, as described in the California Evaluation Protocols, and spelled out in greater detail by OPower's testimony, is well equipped to deal with certain analytical issues and policy concerns raised by the overlap of the savings targeted by comparative energy use reports, and programs already underway through Commission directive.
28. The policy determination in D.09-09-047 to freeze ex ante values could potentially lock in overly optimistic projections for behavior-based programs.
29. Results from the final 2006-2008 evaluation reports can be used as inputs for calculating the energy impacts of 2009 programs for those programs that were evaluated during the 2006-2008 period and also extended during 2009. Using these results would conserve resources and ensure consistency with the evaluation of the 2006-2008 programs from which the 2009 bridge-funding portfolios were derived.
30. Concerns about counting of Codes and Standards savings pre-2006 have been sufficiently resolved to allow 100% of savings to be counting toward energy savings goals.
1. Proper notice was given to parties that the Commission may in this proceeding adopt changes that would modify D.05-01-055.
2. The Joint IOU/ED EM&V Plan is reasonable and should be adopted.
3. Experience with EM&V since D.05-01-055 requires that the Commission strengthen ED's management role for EM&V in order to minimize conflicts of interest, reduce duplication, and ensure transparency of information.
4. D.05-01-055 should be modified to provide specific oversight responsibilities for ED's management role for EM&V.
5. ED's management role for EM&V should include timely responses to IOU requests, including specific timeframes.
6. A budget of $125 million for EM&V for 2010 through 2012 is reasonable.
7. OP 42 of D.09-09-047 should be corrected to use the current program funding proportions for the 2010 through 2012 energy efficiency program cycle to determine EM&V funding for each utility.
8. $18.75 million of the $125 million EM&V budget for 2010 through 2012 should be allocated to the IOUs to maintain staffing levels.
9. It is reasonable to expect that IOUs should perform some or all of the types of EM&V studies that they have performed in the past.
10. $34.3 million of the $125 million EM&V budget for 2010 through 2012 should be allocated to the IOUs to perform EM&V studies (including staff costs).
11. It is reasonable for disputes regarding complex and controversial EM&V matters to be considered by the formal decision-makers in the Commission, after attempts at informal resolution.
12. The ED's recommendations as laid out in part of Section C of the ED Straw Proposal for IOU involvement in ED EM&V studies are reasonable.
13. It is reasonable to delegate to ED the task of determining the level of public vetting for EM&V projects.
14. ED's recommendation that all EM&V-related projects, regardless of funding source, adhere to the same policies and procedures as EM&V funded projects is reasonable, with the caveat that the EM&V processes should not apply to projects not previously considered to be in the EM&V category.
15. While ED and the IOUs should work with customers on evaluation surveys to both ensure necessary cooperation and to limit the burden on customers, there is no need to change any specific policy in this area.
16. It is reasonable to allow case-by-case exceptions to the Commission's firewall policy adopted in D.05-01-055 in order to recruit program implementers to collect data needed for EM&V.
17. It is reasonable to adopt a new set of electric and gas avoided costs for energy efficiency resources, including an updated GHG adder of $30/tonne in 2009 using generation cost inputs from the most recently adopted Market Price Referent as of the date of this Decision, and updating the natural gas cost forecast using data as of the date of this Decision.
18. It is reasonable to credit savings from certain proven behavior-based programs, using methodologies contained within the California Evaluation Protocols.
19. It is reasonable to use the results from the final 2006-2008 evaluation reports as inputs for calculating the energy impacts of 2009 programs, for those measures and programs that were evaluated during the 2006-2008 period and also extended during 2009.
20. 100% of pre-2006 Codes and Standards savings should be counted toward IOU energy savings goals.
IT IS ORDERED that:
1. Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company shall adhere to the "2010 - 2012 Joint Energy Division and IOU Evaluation Measurement and Verification Plan" in Attachment A.
2. The budget of $125 million for Evaluation, Measurement and Verification (EM&V) for 2010 through 2012, tentatively adopted in Decision 09-09-047, is affirmed. A party seeking to increase the 2010-2012 EM&V budget may file a Motion in Rulemaking 09-11-014, the open energy efficiency Rulemaking.
3. Ordering Paragraph 42 of Decision 09-09-047 is corrected to read: "An initial Evaluation, Measurement and Verification budget of $125 million is adopted, subject to review in the follow-up Evaluation, Measurement and Verification decision in this docket. $88 million in remaining funds shall be used for these purposes, with $37 million in additional funds approved for 2010-2012. Evaluation, Measurement and Verification funds shall be allocated as follows: Pacific Gas and Electric Company 43%; Southern California Edison Company 39%; San Diego Gas & Electric Company 9%; and Southern California Gas Company 9%."
4. The process for Evaluation, Measurement and Verification for the 2010 through 2012 energy efficiency portfolios adopted in Decision 09-09-047 for Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company (collectively, IOUs) shall supersede the process adopted in Decision 05-01-055 regarding the following processes:
· An IOU shall seek approval from Energy Division before initiating Evaluation, Measurement & Verification (EM&V) ex-ante studies, or EM&V process or formative evaluations. The IOU management role for developing ex-ante savings estimates or EM&V process or formative evaluations shall be under the oversight of Energy Division, who shall have the authority to deny approval of projects. This authority is limited to situations where there is a conflict of interest with a contractor the IOU wishes to hire, where there is duplication or significant overlap with studies already planned or carried out by Energy Division, or where Energy Division can specify why a study is unnecessary. Energy Division's approval process for IOU's ex-ante studies, or EM&V process or formative evaluations, is limited to no more than two weeks. Any Energy Division denial of approval shall be in writing to the IOU requesting approval.
· If Energy Division expects to take three months or more to complete an ex ante estimate study, Energy Division shall approve an IOU request to develop ex-ante estimate in order to ensure timely information, or reject the request by providing the IOU, within two weeks of the IOU's request, with a written statement indicating that such rejection is due to duplication, conflict of interest or other specific rationale.
· Review of completed IOU workpapers regarding ex-ante savings estimates are subject to Energy Division review and approval, as set forth in an Administrative Law Judge Ruling of November 18, 2009 in Application 08-07-021, et al. Each IOU shall cooperate with Energy Division to allow upfront consultation regarding such workpapers.
· Energy Division's role for approval and involvement in IOU EM&V projects shall be as set forth in Attachment 2 of this decision.
· Energy Division may make case-by-case exceptions to the Commission-adopted firewall policy regarding program implementers in order to collect data needed for EM&V.
5. Decision (D.) 06-06-063 is modified to adopt a new set of electric and gas avoided costs for energy efficiency resources, including a greenhouse gas adder of $30/tonne, using generation cost inputs from the most recent Commission-adopted Market Price Referent as of the date of this order. Energy Division shall update the natural gas avoided cost for energy efficiency resources using natural gas price data as of the date of this order.
6. A total of $34.3 million for Evaluation, Measurement and Verification (EM&V) studies for 2010 through 2012, including staffing costs, allocated among Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company consistent with Ordering Paragraph #3 of this order. Following the process in Attachment 2 to this order, Energy Division shall determine if other EM&V funds shall be allocated to these utilities.
7. A party to Rulemaking (R.) 09-11-014 may file a "Motion for Evaluation, Measurement and Verification Dispute Resolution" (EM&V Motion) with the assigned Administrative Law Judge for resolution of an EM&V matter. The EM&V Motion must include a statement from Energy Division giving its side of the dispute and documentation of an attempt at informal dispute resolution. The Administrative Law Judge may issue a Ruling to resolve the dispute.
8. In a Motion for Evaluation, Measurement and Verification Dispute Resolution filed pursuant to Ordering Paragraph 7 of this order, the filing party or the Energy Division may ask that the matter be resolved by the assigned Commission or the full Commission. In that case, the Administrative Law Judge (ALJ) will consult with the assigned Commissioner to determine the appropriate course of action. In this situation, the assigned Commissioner or ALJ may issue a Ruling to resolve the dispute. If the assigned Commissioner determines the matter should be brought before the full Commission, the ALJ or assigned Commissioner shall issue a Proposed Decision and allow for comment under Rule 14 of the Commission's Rules of Practice and Procedure.
9. A Motion for Evaluation, Measurement and Verification (EM&V) Dispute Resolution filed pursuant to Ordering Paragraph 7 of this order may be used for the following purposes only:
· Dispute over selection of an EM&V contractor;
· Disputes about project-specific final EM&V work plans;
· Disputes over results of EM&V studies or reports (except for Energy Division Verification Reports, which are issued via draft resolutions per D.08-12-059);
· Disputes regarding final EM&V technical reports; and
· Disputes concerning public vetting of EM&V projects.
10. The process for Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company (investor-owned utilities or IOUs) involvement in Energy Division (ED) Evaluation, Measurement and Verification (EM&V) studies shall supersede the process adopted in Decision 05-01-055, and shall be as follows:
· ED and the IOUs will convene publicly-noticed meetings among their staff, EM&V contractors, and stakeholders to share key results and EM&V findings that might lead to improvements in the portfolio and identify best practices and possible improvements to evaluation methods. Such meetings will take place sometime around the middle of the program cycle or at such time when significant results from various EM&V projects are available. If so requested by parties or stakeholders, ED or IOUs, or both, should hold short informal meetings with groups or individual organizations, to discuss EM&V work progress and results.
· ED and IOUs will convene ad hoc meetings (approximately quarterly) among ED staff, EM&V contractors, IOU EM&V staff and IOU program managers to discuss work progress and results. These meetings are to provide for timely feedback to program design and implementation. The IOUs can request meetings with ED to discuss work progress and results at any time.
· When significant results are produced by the EM&V work, and a technical report is not immediately pending, the ED and/or the IOUs will provide informal written summaries of the results to the IOUs and other stakeholders. These written summaries will be posted on the same website used for posting EM&V work plans and comments.
11. Energy Division shall determine which Evaluation, Measurement and Verification projects should be publicly vetted, and shall follow the process laid out in the Energy Division Straw Proposal, pages 8-11, issued by Ruling in this proceeding on July 7, 2009.
12. All Evaluation, Measurement and Verification (EM&V) - related projects undertaken by Southern California Edison Company, Southern California Gas Company, San Diego Gas & Electric Company, and Pacific Gas and Electric Company, regardless of funding source, shall adhere to the same policies and procedures adopted in this Order as EM&V-funded projects, except that such EM&V policies and procedures do not apply to projects not previously considered to be in the EM&V category.
13. Savings from behavior-based energy efficiency programs, defined as comparative energy use reporting contemplated in Senate Bill 488, shall be eligible for counting, if evaluated consistent with experimental design methods contained within the California Evaluation Protocols. The prioritization process described in the Joint Energy Division/Utility Plan for energy efficiency Evaluation, Measurement & Verification in 2010 through 2012, adopted in Ordering Paragraph 1 of this decision, shall be used to make decisions regarding which programs will be evaluated and specifically how those programs will be evaluated.
14. Savings for behavior-based energy efficiency programs shall be credited solely on an ex post basis.
15. Results from the final 2006-2008 evaluation reports shall be used as inputs for calculating the energy impacts of 2009 programs, for those measures and programs that were evaluated during the 2006-2008 period and also extended during 2009. The energy impacts of the 2009 programs shall be reported by Energy Division before the end of 2010, or as otherwise required in Rulemaking (R.) 09-11-014, R.09-01-019, or other applicable energy efficiency docket.
16. 100% of pre-2006 verified savings from Codes and Standards advocacy work shall count toward achievement of Commission energy savings goals for the 2010 through 2012 energy efficiency program cycle. Verified Codes and Standards savings pre and post-2006 shall count only for savings within the utility's service territory.
17. Applications (A.) 08-07-021, A.08-07-022, A.08-07-023, and A.08-07-031 are closed.
This order is effective today.
Dated , at San Francisco, California.
ATTACHMENT 1
2010 - 2012 Joint Energy Division and IOU Evaluation Measurement and Verification Plan
1. Introduction and Scope of Joint EM&V Plan
Energy Division (ED) and the Investor-Owned Utilities (IOUs) submit this Joint Evaluation, Measurement and Verification (EM&V) / Policy and Planning (PP)23 Plan ("EM&V Plan") pursuant to Commission Decision 09-09-047, "Decision Approving 2010-2012 Energy Efficiency Portfolios and Budgets," issued on October 1, 2009. The EM&V Plan represents a cooperative effort by ED, Southern California Edison (SCE), Southern California Gas (SCG), San Diego Gas and Electric (SDG&E), and Pacific Gas and Electric (PG&E) staff to present a joint EM&V planning proposal and budget for the 2010-2012 energy efficiency portfolios authorized in Decision 09-09-047.
In Decision 09-09-047, the California Public Utilities Commission ("CPUC" or "Commission") addressed certain EM&V issues and policies and deferred resolution of others to a subsequent EM&V Decision. In anticipation of the subsequent EM&V decision, the Commission ordered ED and the IOUs to prepare an EM&V plan to be jointly submitted to the assigned ALJ and issued for comment via ruling. The plan presented herein is responsive to the Commission's stated desire "...to make near-term improvements in order to streamline EM&V processes, and enhance timeliness, transparency and consistency across EM&V work products" (D. 09-09-047 at p. 301) and "to take a fresh look at several aspects of our EM&V activity in California for the upcoming program cycle, to reduce unnecessary burden on staff and other resources, and streamline our EM&V processes." (D.09-09-047 at p. 294).
In D.09-09-047, the Commission adopted "1) a budget for 2010-2012 EM&V, 2) Commission goals for EM&V, and 3) a process for adopting detailed EM&V projects, refined EM&V budgets, and remaining EM&V policy issues in a subsequent EM&V Decision expected in the final quarter of 2009." The Commission clarified that the subsequent EM&V decision would include, but would not be limited to the following issues:
o Approval of the joint Energy Division and utility EM&V plans and Budgets
o Clarification of the respective scope of responsibilities for IOU and ED staff
o Recommendation on improved stakeholder input process for EM&V projects
o Improvements to the cost-effectiveness calculation tool and tracking and reporting requirements for EM&V related data
o Frequency and Scope of DEER Updates
o Consideration of methodologies to verify savings driven by behavior based energy efficiency programs
(D.09-09-047 at pp.301-04)
This EM&V Plan presents the ED and IOU joint proposals with respect to EM&V planning and budgeting.
2. Guiding Principles
The EM&V plan is guided by the Commission's Goals for EM&V, as articulated in Decision 09-09-047:
"EM&V activities shall be planned and implemented to achieve the following core objectives in order to support the Commission's oversight function of ensuring the efficient and effective expenditure of ratepayer funds within the energy efficiency portfolios. All activities should be undertaken to meet the overarching goals of clarity, consistency, cost-efficiency, and timeliness.
The core objectives are:
1. Savings Measurement and Verification - Measurement and verification of savings resulting from energy efficiency measures, programs, and portfolios serve the fundamental purpose of developing estimates of reliable load impacts delivered through ratepayer-funded efficiency efforts. Measurement and verification work should reflect a reasonable balance of accuracy and precision, cost, and certainty, and be designed for incorporation into in procurement planning activities.
2. Program Evaluation - Evaluation of program-specific qualitative and quantitative measures, such as the program performance metrics discussed earlier in this decision and process evaluations, serves a key role in providing feedback for the purposes of improving performance and supporting forward-looking corrections to utility programs and portfolios. In order to maximize return on ratepayer dollars, program evaluations must be completed on a timeline which informs mid-course corrections and/or program planning for the following cycle.
3. Market Assessment - In a constantly evolving environment, market assessments are an essential EM&V product needed to set the baseline for strategic design and improvement of programs and portfolios. Saturation studies, surveys of emerging technologies and other such analyses which inform estimates of remaining program potential and forward-looking goal-setting are key aspects of market assessment.
4. Policy and Planning Support - Consistent with prior program cycles, it is essential to reserve funding to support overarching studies and advisory roles which support Commission policy goals. Over the last program cycle this has been inclusive of potential and goals studies, maintenance of DEER database, developing databases of best practices for program design and delivery, program design mix, and other means which support the Commission's oversight role, but do not fall under the core EM&V categories described above.
5. Financial and Management Audit - Supporting the Commission's oversight function of ensuring the efficient and effective expenditures of ratepayer funds within the utilities' energy efficiency portfolios is another objective of EM&V activities. Rigorous financial and management audits overseen by Commission staff will be critical in ensuring that the utilities' general and administrative costs, and other program expenditures are prudent and reasonable.
3. Informal Goals and Guiding Principles for Increased ED/IOU Collaboration
Staff from ED and all four IOUs met for fours days of working meetings to develop EM&V budgets and a joint EM&V plan. During these meetings we agreed that we needed to develop a more collaborative and transparent working relationship as an important step towards improving the EM&V process. "Collaboration" in this document is defined as IOU and ED staff working together on shared EM&V projects, as well as working on separate EM&V projects following mutually agreed upon standards for transparency, respect, and communication. We believe that this more collaborative process will result in greater cost-efficiencies, more reliable results, broader stakeholder buy-in, and fewer disputed issues.
The following are informal goals for the purpose of fostering a working relationship built upon mutual respect and transparency. They represent an informal Energy Division and IOU staff-level agreement on general principles to guide staff-level collaboration and interaction on EM&V projects. These are not intended to impose formal or specific obligations on the ED or the IOUs and do not define the formal division of EM&V roles and responsibilities.
a. Transparency
ED and IOU EM&V staff will conduct EM&V projects in a transparent manner wherever possible.
i. Open Communication - In order to cultivate better collaboration and make more productive use of EM&V results, ED and the IOU staff will engage in open and truthful communication regarding EM&V projects.
ii. Regular Communication - ED and IOU staff will hold regular meetings to provide each other updates on their respectively managed EM&V projects. Upon issuance of the decision, ED and IOU staff will collaborate to determine an appropriate schedule for these meetings.
iii. Joint Participation - An effort will be made to include both ED and IOU EM&V staff in all stages of all EM&V projects.
iv. Sharing of EM&V Data and Information - All data and work products resulting from all EM&V projects should be made available to both ED and IOU EM&V staff when the data becomes available. Tracking of EM&V Projects - All EM&V projects will be disclosed and tracked in an easily accessible tracking system.
v. Timeliness - Communication regarding EM&V plans and results, and actions based on those results, will be conducted in a timely manner.
b. Consensus
Although ideal, it may not always be possible or productive to reach consensus between ED and IOU staff during the planning and implementation of EM&V projects or interpretation and use of EM&V results. ED and the IOUs will seek to achieve consensus through informal processes. If consensus cannot be reached informally, ED and the IOUs will follow the applicable dispute resolution processes in effect wherever a formal resolution is necessary.
c. Cost-efficiency
ED and the IOUs will plan and implement EM&V projects with the goal of achieving the highest benefit for the EM&V expenditure, while minimizing interference with the programs and utility customers. To that end, EM&V projects will be designed to avoid duplication of effort, consolidated across all activities in a streamlined manner, and planned to comply with the Commission's schedule requirements. This document and the proposed budget allocations represent ED and the IOUs' current best judgment on the appropriate allocation of the authorized EM&V budget to EM&V projects needed to accomplish technically credible, quality work products that will comply with the Commission's requirements and goals stated in Decision 09-09-047.
d. Time for collaboration
EM&V projects will be scheduled in a manner that allows sufficient time for input and participation between ED and IOU staff, as well as other stakeholders as required by the Commission. ED and the IOUs recognize that the time needed to implement and complete EM&V projects may generally be longer as a result of this effort. ED and the IOUs also recognize the possibility that the desired time allotment for collaboration on some EM&V projects may not be possible due to superseding requirements, such as adherence to Commission mandated schedules or the need to take advantage of a time sensitive field situation.
e. Ethical standards and technical best practices
EM&V projects will always be conducted in accordance with the laws of the State of California, Commission established policies, and Commission adopted technical standards (such as the California Evaluation Protocols). Additionally, ED and the IOUs intend to manage EM&V projects following guidance from the relevant professional societies' standards for ethics and technical best practices. Such standards include the International Performance Measurement and Verification Protocols (IPMVP); the American Evaluation Association's Guiding Principles for Evaluators;24 the National Society of Professional Engineers Code of Ethics for Engineers;25 and the American Society of Heating, Refrigerating, and Air-Conditioning Engineers Code of Ethics.26
4. EM&V Planning Framework
At the time of this EM&V Plan, the 2010 - 2012 IOU portfolios are just recently adopted by the Commission, and program plans are expected to undergo additional refinement over the next four to six months through the final stages of the program planning process and compliance filings ordered by Decision 09-09-047. Additionally, as the adopted EE portfolio is implemented, program plans will necessarily evolve to adapt to changing circumstances, program funding may be shifted around, new programs may be designed and fielded, and some programs may be terminated. For these reasons, ED and IOU staff have agreed that the optimal EM&V plan for the Commission to adopt at this time is an EM&V Planning Framework guided by existing Commission policy and ED & IOU staff experience and expertise that gives EM&V the needed flexibility, rather than a detailed plan that makes assumptions about the full scope of EM&V needs over a three year period. This document outlines the basic elements of this proposed EM&V Planning Framework, and ED/IOUs jointly request party input primarily on this proposed Framework and proposed areas of work, not on specific draft EM&V/PP project budgets.
Given the many Commission required EM&V projects, multiple possible EM&V needs, and constraints on EM&V staff and consulting resources, there is a need to prioritize and optimize across EM&V research areas and individual projects, as well as a need to plan and implement EM&V project in phases. To accomplish the next three years worth of EM&V as effectively and efficiently as possible, ED and IOU EM&V staff propose the following EM&V Planning Framework:
a. Phased and ongoing project planning and transparent project implementation
ED and IOU EM&V staff believe it is necessary to commit funding, staff, and consulting resources to fully planning and implementing EM&V projects in phases and as priorities change, rather than to develop plans for all EM&V projects first and then subsequently implement all EM&V projects, as was the practice for impact evaluations during the 2006 - 2008 cycle. ED and the IOUs therefore recommend that the Commission adopt a process that provides the EM&V staff with the flexibility to plan and implement EM&V in phases and on an ongoing basis, rather than committing to a three year plan for all EM&V projects at this time. In lieu of a full three year EM&V plan, ED and the IOUs recommend that the Commission clearly articulate standards for transparency and stakeholder participation which ED and the IOUs will follow during the course of the 2010-2012 EM&V projects.
b. Continuous optimization and prioritization of EM&V work
One of the first projects that ED and the IOUs will initiate is a review of previous EM&V work, particularly EM&V projects conducted during 2006-2008, and a gap analysis to optimize EM&V activities and determine priorities across all EM&V research areas. The gap analysis will first create an inventory of recent and ongoing EM&V projects and compare this information with EM&V needs moving forward. Continuous optimization will be done via an ongoing status review and prioritization exercise facilitated by the phased implementation of EM&V projects, as well as flexibility in EM&V planning and implementation that allows EM&V to be responsive to contextual changes over time. EM&V projects will emphasize the flow of EM&V findings to program managers for the purpose of program improvement.
c. Integration of EM&V projects across functions
One of the goals of the proposed EM&V planning framework is to look for opportunities to create synergy and integration across different EM&V research objectives and needs, rather than viewing individual needs as disparate elements competing for limited resources. This includes avoiding the creation of unnecessary silos of EM&V activities, avoiding unnecessarily duplicative data collection and analysis, and identifying ways in which EM&V can be organized and implemented to meet multiple needs in a cost-effective manner.
5. Initial EM&V Plan
As discussed above, ED and the IOUs plan to design and implement EM&V in phases by order of project priority. The First Phase of EM&V projects is work that needs to be immediately initiated in order to set up a more efficient EM&V "infrastructure" that makes cost-effective improvements on the use of all EM&V resources, data, and processes. The First Phase will also include some research projects that are immediately needed by the IOUs in order to make rapid adjustments to the new program portfolio. ED and the IOUs intend to begin work on projects in this First Phase using previously approved 2009 bridge-funding for EM&V. The First Phase projects will need to begin prior to a final Commission Decision approving EM&V plans and budgets, hoped for in late 2009. Several of the First Phase projects will be ongoing and may continue through 2012. The Second Phase and Third Phase projects are briefly describe at the end of section 5.
ED and the IOUs expect the First Phase to consist of the following projects:
a. EM&V inventory, priority analysis, and gap analysis
Upon completion of this draft EM&V plan, ED and the IOUs will immediately begin a review of EM&V work conducted since 2004 for the purpose of creating an inventory of recent EM&V work. This analysis will include an assessment of the quality and usefulness of the research products.
A simultaneous effort will be undertaken to create a similar inventory of research required by the Commission in D. 09-09-047 and anticipated as needed by ED and the IOUs. The IOU EM&V staff will discuss potential immediate research needs with the IOU program staff and will develop research proposals for statewide and utility specific EM&V projects. ED staff will develop proposals for research projects that are considered necessary in order to accomplish the Commission's EM&V goals stated in D. 09-09-047 (pgs. 299-301), as well as other specific research goals mandated by the Commission in that Decision.
ED and the IOUs will compare potential projects to identify areas where efficiencies can be gained by consolidating projects. The proposed projects will then be placed in an order of priority according to criteria such as uncertainty of program impacts, relative magnitude of impacts, future program and market potential, and implicit or explicit importance attributed to the project by the Commission.
The prioritized list of projects will then be compared to the inventory of previous EM&V research to determine where the previous research can be used in lieu of conducting new analysis, or where the previously completed research can be leveraged to make any new analysis more robust and reduce costs. Once this gap analysis step is completed, the prioritized list of projects will be updated and the projects will be implemented in order of priority.
Expected Timeline
2009 December - 2010 January Review and inventory of recent EM&V work
2010 January - February Inventory of proposed research
2010 February-March Project consolidation and prioritization
2010 March-April Gap analysis and revised priorities
b. Reporting standards and reporting tools improvements.
In D.09-09-047 the Commission requested that Energy Division "...review further all existing and new energy efficiency reporting requirements and report on possible ways to streamline these requirements." Additionally, the IOUs were specifically instructed to report gross savings relative to goals, progress towards accomplishing performance metrics goals, and additional information related to the administration of local government partnership programs. Energy Division has also begun the development of a new cost effectiveness tool (CE Tool) for the purpose of enforcing data quality controls, streamlining the review of cost effectiveness inputs, and making the IOUs' savings reports directly traceable to the program tracking systems.
To accomplish these objectives, ED will conduct a review of all energy efficiency reporting requirements, existing and planned reporting tools related to EM&V and will prepare and implement a comprehensive plan to create a reporting system that is more streamlined, cost-effective, and useful.
Expected Timeline
2010 January - 2010 March Review of existing reporting requirements
2010 March ED report on reporting requirements
2010 April Commission action on ED report
2010 May Onward Implementation of new reporting requirements
c. Program evaluability assessment and data collection enhancements.
This project will take initial steps towards addressing the issues specifically raised in Attachment C of the July 7th ALJ Ruling issued in A. 08-07-021. While this project will focus specifically on improvements to the data systems needed for conducting the detailed EM&V work, it is related to, and will be done in conjunction with the reporting standards and reporting tools improvements project described above. The project will consist of a review the data availability problems encountered in the 2006-2008 program evaluations and the program designs and tracking systems in place for the 2010-2012 program portfolios. The project will aim to develop data collection and data transfer protocols as well as tracking system enhancement procedures. The goal is to assure that the data needed for EM&V will be efficiently available to the ED and IOU EM&V staff and their contractors.
Expected Timeline
2010 January - 2010 March Review of existing data systems
2010 April Comprehensive long-range data management plan
2010 May Data collection and data transfer protocols
2010 May Onward Implementation of data system improvements
d. Development of a process for integrating project inspection, M&V, and process evaluation for larger program participants.
IOUs require the largest customized nonresidential projects to submit to pre-project and post-project implementation inspections. While there are comparatively few of these larger projects, the expected savings from these projects are quite significant, which means they are more likely to be sampled for both M&V and process evaluations. Furthermore, the larger projects are typically quite complex, requiring more time for the utility inspectors and ED evaluators to be on the participant's site.
From the participant's perspective, the impact of multiple seemingly redundant inspections can be an unanticipated burden; and from the CPUC's perspective, this redundancy can be a potentially inefficient use of ratepayer expenditures. ED and the IOUs will work cooperatively to make the evaluation of large energy efficiency projects more cost-effective and less disruptive to the customer by jointly developing procedures to identify large projects early and include the ED evaluators in the pre- and post- project inspection process. ED and the IOUs will make efforts to consolidate evaluation surveys so as to minimize customer inconvenience.
Expected Timeline
2010 January - February Review of IOU project inspection procedures
2010 March Review of expected evaluation needs
2010 May Integration plan
e. Development of plans to gather necessary market baseline data.
This project will identify key market indicators that have been or can be influenced by a program intervention. The primary purpose of this early market baseline project is to develop and implement a work plan that provides a basis for later comparisons of the status of the key markets after program intervention, in order to help assess the impact of the program or programs. More than one program can impact a market, and a program can impact more than one market. ED and the IOUs will therefore carefully identify the "markets" that need to be researched early in the cycle, with timely collection of appropriate market data that will serve as the baseline.
The evaluation needs to address the period over which the market effect will remain, the level of effect experienced in the market over time, the degree to which the program's efforts caused the market effect, and the amount of energy savings provided by the effect. All this presumes the appropriate market data collected before the program effects occur.
In general, any key market indicator that the program theory predicts will be changed by the program should be considered for inclusion in either a market characterization or a baseline study. However, markets are constantly in a state of change, so we will seek to identify not only those market indicators that are important under the initial program theory, but also those that could become important later.
Steps in this Activity
1. For each program, identify key market indicators that the program theory predicts will be changed by the program.
a. Clearly define the targeted market(s);
b. Develop a detailed description of market operations and factors that contribute to their status;
c. Describe the market hypothesis on which the various program activities and the expected effects rely;
d. Describe the baseline condition that is expected to occur without program intervention including impacts from other external factors that affect the market; and
e. Describe the causal linkages that lead from program activities to the accomplishment of the program's goals in isolation from other external market effects.
2. Develop sets of indicators for each market that will provide efficient but effective measurements of the identified market effects. Include a plan to isolate external market effects in order to gain a true perspective of identified program related market indicators.
3. Collect baseline values for these indicators, before the program effects begin to take place.
Expected Timeline
2010 January Develop scopes of work for RFPs
2010 February Issue RFPs and select contractors
2010 March-February Analyze statewide programs
2010 May Develop indicator systems
2010 June onward Collect data for indicator systems
f. Procurement of management and technical consulting services for ED.
Given the complexity and workload involved in managing EM&V projects, ED staff expect to rely on a team of expert consultants to assist with oversight, management, and advisory functions. ED will conduct competitive solicitations for this work towards the end of 2009 and expects to have a new team of consultants available to assist ED in early 2010.
Expected Timeline
2010 January Develop scopes of work for RFPs
2010 February Issue RFPs
2010 March Select consultants
g. Development of a detailed plan for ED and IOU coordination.
Once the upcoming EM&V Decision is finalized, ED and the IOUs will develop a coordination plan that incorporates and implements the policies, rules, and any specific projects required therein. The coordination plan will be a guidance document for ED and IOU staff and can be made publicly available if the Commission desires. We expect the coordination plan to be completed within 30 days of the final EM&V Decision. In the meantime, ED and the IOUs will prepare an interim coordination plan which sets out our mutually agreed process for collaborating on First Phase projects.
Expected Timeline
2009 January -February Interim ED/IOU coordination plan
2010 April - May Final ED/IOU coordination plan
h. Behavioral Energy Savings estimation methods
In D.09-09-047 the Commission indicated its intent to "...consider expedited approval of new EM&V methodologies to verify savings driven by behavior-based Efficiency programs (currently considered non-resource programs)." And that the EM&V work "...should ensure synergies and leveraging of any new behavior-based approaches with the residential programs approved herein..." On October 11, 2009, the Governor signed SB 488 into law, which requires the CPUC to evaluate certain residential benchmarking programs using an experimental design approach.
Energy savings from behavior programs have traditionally been measured with a billing or consumption analysis. ED and the IOUs will review best practices in billing analyses as well as the data requirements that would be needed to support robust billing analyses. ED and IOU EM&V staff will coordinate with program managers for IOU residential benchmarking programs to ensure that participants are randomly assigned to treatment and control groups (or comparison groups) to ensure that an experimental design billing analysis can be completed, at least on a pilot basis. ED has been conducting a review of different types of behavior intervention programs as well as some pilot EM&V projects and will extend this effort into full program evaluations wherever it makes sense to do so. ED and the IOUs will also work on ways to tightly coordinate the delivery and evaluation of behavior based energy efficiency programs with the ongoing advanced meter infrastructure roll-out and eliminate any redundant and overlapping efforts.
Expected Timeline
2009 February Review of billing analysis methods and data requirements
2010 March - April EM&V protocols and methods for behavior programs
i. IOU market assessments, early EM&V, and process work
The first phase projects include IOU projects that need to start immediately due to the time-sensitive nature of data collection as well as the criticality of information needs for program implementation in 2010. All the IOU Phase 1 projects will be conducted in collaboration with the Energy Division.
Early M&V:
· Assessment of savings from server virtualization and data centers.
· Window film lifetime and replacement practices.
· Lighting baseline usage for selected applications lacking this data (preschool, pool lighting etc).
· Pool pump usage.
· Remaining Useful Life assessment for selected technologies.
· Heating/Ventilating/Air conditioning maintenance savings - Phase 1 (secondary research) and Phase 2 (controlled/quasi-experimental research).
· Retrocommissioning measure energy savings/prediction tools research for most common measures.
· Single family residential new construction energy savings and incremental measure cost update.
Market Assessment:
· Advanced Lighting Market Assessment - to help guide the Advanced Lighting program and provide information on the current state of the market for technologies such as LED as well as the ways in which the market is changing both from the supply side and the demand side.
· Market Study of Deep Energy Reduction for the Whole House Market - to assess the availability of infrastructure and technologies to help in planning, market awareness, knowledge and acceptance of deep energy reduction strategies, and how to overcome these barriers for homes.
· Market Study of Water Energy Savings -to assess the current penetration/potential for water energy savings technologies within the residential and commercial segments.
· Plug Load Market Potential Study - to assess the size of the market opportunities (will be coordinated with any Energy Efficiency Potential Study update plans).
· Industrial EE Program Market Assessment Study for customers with either high gas and/or electric loads- to determine customer sectors most sensitive to the current challenges and opportunities.
· Residential New Construction Customer Decision Study -to assess the "decision triggers" and current levels of awareness of the various existing carbon/low energy labels and associated lower energy home opportunities.
· Market Assessment on Code Compliance - to identify areas of weak code compliance and highlight market barriers that can be addressed through the Compliance Enhancement Sub-Program.
· Baseline Studies for Partnership Programs - to document existing practices and characterize the needs of the customers and their likelihood of program participation.
· Strategic Industrial Research - to look at market segmentation/dynamics and decision making processes for energy efficiency projects.
· The "Invisible" (Hard-to-Reach) Data Centers - to assess where opportunities exist and develop program strategies to reach these opportunities.
· Pool Vendor Market Assessment - to inform training strategies.
· Agricultural Market Assessment and Energy Efficiency Potential.
Process Evaluation:
· Evaluability Assessments for selected smaller programs not covered by Energy Division's Program Evaluability Assessment Project, to determine if the program outcomes are sufficiently well defined and measurable via data tracking processes, as well as identify any early M&V needs.
· Detailed Program Theory and Logic Model - development where needed for the programs.
· Enhanced Inspection Plan - development for selected programs, including baseline documentation requirements.
· Rapid Feedback Evaluations - for selected programs and specific marketing activities to help provide early feedback and recommendation on program design changes (initial list includes: new construction programs, WE&T, and ME&O targeted marketing campaigns).
· Effectiveness impact of behavioral energy use "peer comparisons" tools (i.e. Home Energy Reports) for residential customers.
· Impact of In-Home Displays: to drive customer participation in EE, effectiveness of marketing and outreach activities associated with real-time usage data.
· Cost effectiveness of solar water heating technologies in utility programs.
Timeline Varies by Specific Project
j. Portfolio Cost Effectiveness Methodology
This category of analysis will include a review of existing cost-effectiveness methodologies and development of new methodologies that seek to measure cost effectiveness at the program and portfolio level. This methodology should reflect the California Long-Term Energy Efficiency Strategic Plan goal for market transformation by not only considering program costs in relation to savings realized, but also include an analysis of program costs in relation to market transformation objectives and goals.
Timeline TBD
k. Goals/Potential analysis
Analytic consistency is an essential starting point in setting aggressive yet realistic goals for EE programs while also developing "stretch" goals for energy efficiency savings. Setting stretch goals require a consideration of additional technologies, measures, and savings potential available to the utilities but not reflected in the current potential study informing current goals. This category of analysis will reexamine goals and potential to inform the development of stretch goals while at the same time not reducing the rigor by which current goals exist. This analysis will inform forward-looking goals on the basis of updates to measure savings parameters.
Timeline TBD
A Second Phase of projects will be planned and implemented as soon as assignments are made and work is underway on the First Phase projects, but no later than the first quarter of 2010. The Second Phase projects will include the formative M&V, process evaluation, and market research that is needed to provide early assessments of the programs and make decisions about program modifications, but which were not launched as part of the First Phase. We anticipate that the Second Phase projects will be initiated during the first and second quarters of 2010.
Finally, a Third Phase of EM&V projects will be planned and implemented when ED and IOU staff are convinced that Second Phase projects are successfully underway and likely to achieve project goals. The Third Phase projects will primarily be the summative, or ex-post, evaluations that have been employed by the Commission to establish retrospective statements of portfolio accomplishments. Additional formative work may also be implemented during the Third Phase of EM&V projects, if needed. We anticipate that the Third Phase projects will be initiated between the second and fourth quarters of 2010, after the Commission rules on the incentive mechanism for 2010-2012 in Rulemaking 09-01-019.
6. Proposed EM&V Budget
Below we present the proposed allocation of authorized budget for all ED and IOU EM&V projects, as well as ED staff Policy and Planning projects. In D.09-09-047, the Commission indicated a desire to keep the EM&V budget at 4% (approximately $125 million) with the expectation that the ED and IOU EM&V staff can produce cost efficiencies and streamline the scope and reporting of EM&V projects. While the Commission also indicated a possibility that it would consider changes to the initial EM&V funding based on proposals for additional funding brought forth in the EM&V plan, ED and the IOU EM&V staff have taken the Commission's desire to manage costs seriously and will strive to complete a robust research portfolio for under $125 million.
While we are confident that the authorized budget will be sufficient to complete a reasonably comprehensive set of EM&V projects, the range of studies needed for 2010-2012 is substantially greater than the range of studies completed for 2006-2008. We are therefore compelled to emphasize that some potentially important research projects may not be implemented if we are to prioritize effectively. Thus, we ask that the EM&V decision keep open the option offered in D.09-09-047 to request more funding if we determine that sufficiently important projects cannot be funded.
2006-2008 were start-up years for both the ED and the IOU EM&V groups, with many start-up difficulties and new systems that did not function optimally. As a result, a number of the important planned studies could not be completed. The experiences of 2006-2008 uncovered some weaknesses in current utility and CPUC tools that need to be strengthened. These include the EM&V structure itself, utility tracking and reporting systems and their ability to meet EM&V data needs, as well as multiple concerns surrounding ex-ante savings parameter updating and documentation, and cost-effectiveness issues. Finally, the adoption of the California Long Term Energy Efficiency Strategic Plan pushes programs, planning, and coordination in far-reaching new directions. This necessitates a major investment in coordinating with market actors and state agencies, policy analysis, and planning for the 2013-2015 cycle and beyond, in each of the strategic areas of focus, and it creates a host of new information needs.
a. Overview and rationale of budgeting process
The IOU EM&V team solicited input from program staff regarding the programs that will be offered during the 2010-2012 cycle. The EM&V team also reviewed the process evaluations, market assessments, and early M&V projects performed during the 2006-2008 cycle to identify additional research requirements. In collaboration with the program staff, the EM&V team compiled a list of market assessment and early M&V needs and process evaluations related to the programs that are being offered in 2010-2012. Based on previous experience, the EM&V team then estimated the cost of performing these studies, including the costs related to the EM&V staff.
ED staff developed budgets for impact evaluations, performance metric evaluations, and overarching and support projects using expert judgment and experience managing similar projects during the 2006-2008 timeframe. These estimates take into consideration expected efficiencies to be gained from the proposed prioritization and optimization process, as well as the fact that the projects will be managed by staff (both IOU and ED) that have gained considerable additional experience managing the 2006-2008 EM&V projects.
b. Request for full fund shifting flexibility
The specific studies and their associated budgets listed in the Table in Section 6c below are ED and the IOU's current estimate of the optimal allocation of the authorized EM&V budget. Section 5 of this plan describes the multi-stage process that ED and the IOUs will go through for determining and prioritizing what studies will be done, when, and with what level of project budget. The process will include making decisions about which organization will contract for each project, who will take primary project management responsibility for it, and the level of involvement of the Commission staff in overseeing each project.
In order to allow ED and the IOUs to respond to changes in the market and to new insights in evaluation, fund-shifting flexibility is needed within the EM&V budget. This includes not only shifting funds between projects, but also, to some extent, between funds managed by the IOUs and those managed by ED, as they mutually agree. ED and the IOUs agree that a minimum allocation of 15% of the EM&V budget to the IOUs is appropriate to support necessary EM&V activities until such time as the Commission issues a final EM&V decision and budget. These costs are currently included as part of the process evaluation, market assessment and early M&V study costs in the budget estimates in Table C. ED and the IOUs were not able to reach consensus as to any further pre-allocation of the remaining 85% of the EM&V budget. ED and the IOUs agree that it is appropriate for the IOUs to include any specific proposals for allocation of the remaining 85% of the EM&V budget in their comments to this EM&V Plan. As during the 2006-2008 cycle, the utilities will be responsible to pay the Energy Division-approved costs for all projects contracted and/or managed by Energy Division.
Each utility's EM&V budget will be its proportional share of the total EM&V budget approved by the Commission, with the proportion equal to its proportion of total program budgets: 43% for PG&E; 39% for SCE; and 9% each for SDG&E and SoCalGas.
This requires correcting Ordering Paragraph 42 of D.09-09-047, which inadvertently used the program funding proportions from the 2006-2008 cycle.
Each utility will pay for its studies that are determined to be acceptable utility-specific studies, out of its overall EM&V budget.
This EM&V fund-shifting flexibility request is consistent with Commission practice for at least the last two decades, and probably for the entire history of EM&V funding for EE programs. The Commission has always recognized the benefits of setting an overall budget but allowing EM&V decision-makers to determine EM&V priorities and budget allocations for the costs to meet them in an ongoing process, rather than assuming that needs and priorities are all known in advance and will be unchanged over a program funding cycle.
c. Budget Table
d. Description of budget categories
The general EM&V project area descriptions below provide summaries of the categories of work used to set the EM&V budget proposed herein. While these project areas are considered necessary preliminarily, they are provided for illustrative purposes and are subject to change as ED and the IOUs continue with the prioritization process. Final research project goals, scope, timing, and deliverables will be determined during development of detailed statements of work included in the contracting process.
EM&V Project Number 1. M&V and Impact Evaluation
M&V is the process of gathering data on energy efficiency technologies and practices from the building and facility where the technology or practice is implemented or typically in use. M&V activities will consist of on-site review and measurement of program activities and energy consumption behavior that can be physically inspected and measured at a customer site or project, as well as the analysis of site level and measure level data through engineering and building simulation models. Site visits will be performed on a probability sample of IOU customers, buildings, or facilities drawn from IOU program tracking databases, IOU billing systems, or the general population. Some M&V data may be collected through remote surveys or by using pre-existing data, if circumstances warrant. Given the enormous scale of energy efficiency program activities, the M&V work will focus on program components selected on the basis of the overall uncertainty of that component's contribution to the total portfolio savings, including potential future savings. These component level evaluations will be conducted at the technology measure level (referred to as high-impact measures, or HIMs). A subsidiary of M&V activity is the physical inspection of installations to estimate measure installation rates.
Impact evaluation consists of evaluation activities designed to measure savings at the program level, such as analyses using utility bill data to produce gross realization rates and net-to-gross studies. Net-to-gross values will be developed for major measure/program strategy combinations and will incorporate reliable attribution for spillover and market effects where data are available and where consistent with Commission policy.
Impact evaluations may also include some indirect impact evaluation activity that addresses those programs or program components primarily designed to obtain behavior changes that eventually lead to energy and demand savings, but not as a direct result of the program intervention. Indirect impact evaluations are used for situations where the primary uncertainty lies in the program's ability to obtain the behavior change targeted by the program. Indirect impact evaluations will therefore be linked to energy or demand savings estimates measured through the HIM M&V, program specific impact evaluations, and/or approved ex-ante estimates.
EM&V Project Number 2. Performance Metrics
Program performance metrics are indicators of the progress of a program toward the short and long-term market transformation goals and objectives in the Strategic Plan. Energy Division developed a process for developing program performance metrics that the utilities shall use when developing these metrics. According to D.09-09-047, the utilities will request approval for their proposed logic models and metrics via an advice letter filing within 120 days of the effective date of that decision. Additionally the utilities will track their program performance metrics using the EEGA or a similar database and will need to develop the tools to submit and track these parameters. The analysis under this category will help complete these tasks.
Market transformation metrics require the identification of indicators to track, the identification of data sources, and agreement on the frequency of data collection, analysis and use. In order to develop these metric recommendations there will need to be analysis on specific market transformation ultimate and proximate indicators, as well as data collection and tracking processes, for a subset of portfolio programs or measures that have the most impact in terms of their importance, such as the Big Bold Programmatic Initiatives, their savings potential or dollars spent. This analysis may consider qualitative factors as necessary and appropriate. It is both necessary and possible to begin the work of gathering baseline data immediately. The IOUs will need to include key data sources and indicators for which to begin collecting market transformation baseline data in their Advice Letters on Utility Program Performance Metrics (see description of performance metric analysis). A process for tracking external market conditions that affect program performance metrics and baseline information will be further developed in the umbrella energy efficiency rulemaking proceeding, or its successor. In that proceeding, we will also consider the appropriate timing for the commencement of the system of market transformation metrics. Market transformation data analysis will inform this effort. Program Performance Metrics and market conditions data serve the following purposes:
· To track California's progress towards achievement of the Strategic Plan objectives, specifically the Big Bold Programmatic Initiatives and other key Plan goals and objectives;
· To inform portfolio development and necessary modifications in future portfolio decisions, including improving program design or eliminating non-performing programs; and
· To target the next generation of improvements and thus continue the cycle of market transformation.
These metrics will be used to track the progress of the programs towards the California Energy Efficiency Strategic Planning market transformation goals.
EM&V Project Number 3. Process Evaluation
The California Evaluation Framework states "a process evaluation is a systematic assessment of an energy efficiency program for the purposes of (1) documenting program operations at the time of the examination, and (2) identifying and recommending improvements that can be made to the program to increase the program's efficiency or effectiveness for acquiring energy resources while maintaining high levels of participant satisfaction." While impact or "summative" evaluations provide an accounting of a program's effectiveness, process evaluations provide insight into program operations that can guide mid-course corrections and future program design.
Process evaluations look at both the program's design and its implementation. This allows program managers to pinpoint where and how whether future effectiveness can be increased by improving program design, program implementation, or both. Process evaluations also provide the valuable function of capturing the story of the program, to share and compare lessons learned with other implementers. Process evaluations can articulate how proximal indicators based on the program's theory (e.g., changes in attitudes) can show whether progress is being made toward long-term goals such as acceptance of emerging technologies. Process evaluations will typically document a program's theory in both detailed narrative form and through a schematic (e.g., "program logic model") that graphically links program resources and inputs to program activities to program outputs to short-, mid-, and long-term outcomes. Process flow narratives and diagrams may also be used to capture program operations, and to identify gaps in program implementation. The logic model and program theory help program evaluators identify gaps in the program's theoretical underpinning, and study these further, to develop recommendations that will likely enhance future success.
Process evaluations use a variety of social science research methods including telephone surveys, in-person interviews, social network analysis, review of program activities and participation data, review of program marketing plans and materials, and field observations. New programs generally undergo an early and more comprehensive process evaluation designed to provide timely feedback on how well the program is being managed and implemented, how well project partners are communicating, and whether initial participants are satisfied with the program's ease-of-use and understandability. Later process evaluations are used to confirm that program design and implementation are still effective.
EM&V Project Number 4. Early M&V Evaluation
Early M&V, managed by IOUs or ED, seeks to validate key savings assumptions and to better understand how savings are achieved for the purpose of improving programs. Early M&V research occurs at the measure-level or parameter-level to:
o Provide in-cycle feedback to programs on savings assumptions
o Correct mutually agreed upon errors in savings estimates
o Improve accuracy of savings estimates for custom calculated projects
o Contribute to future cycle ex ante revisions
o Gather data for developing savings estimates for new measures
o Guide future research to reduce savings uncertainty
Early M&V will be carried out as necessary and results incorporated in program design and planning as soon as feasible.
The Market Assessment studies that will be conducted by ED and the IOUs will include two different study types: market characterization and market baseline measurement.
Market Characterization is a quantitative and qualitative assessment of the structure and functioning of a market, the primary purpose of which is to understand key components and magnitudes of a market, and how the market operates. The study also provides information on how to effectively change the way in which the market functions.
Market Baseline Measurement is the quantification of key market indicators that have been or can be influenced by a program intervention. The primary purpose of the baseline measurement is to provide a basis for later comparisons of the status of the market after program intervention, in order to help assess the impact of the program. This study can also include quantification of size of a particular market so we can monitor the share of market as a result of program intervention.
EM&V Project Number 6. Strategic Plan
This budget category is created to track spending with regard to regulatory support for Energy Division and utility strategic planning efforts. It would include staff time and additional help from consultants. External consultants will provide logistical support with regard to task force / workshop / stakeholder meeting planning, coordination, and staffing. These consultants will work with ED / IOU staff to develop agendas, take meeting minutes, maintain contact information for interested stakeholders, arrange meeting venues and times, communicate with the public, production of task force / workshop / stakeholder meeting material for public dissemination, and all aspects of providing support for task force / workshop / stakeholder meeting planning.
EM&V Project Number 7. Strategic Plan Update Studies
This category of analysis will inform ongoing strategic planning goals and objectives by providing funding for evaluation efforts that are not currently anticipated but will be critical to maintaining continuous forward progress toward meeting these stated goals and objectives. Examples of this kind of analysis are evaluations of portfolio wide leveraging efforts with ARRA investment opportunities that have the potential to allow for the most efficient usage of ratepayer funds while propelling progress towards strategic planning goals and objectives. Other areas that will benefit from evaluation projects not yet fully identified are strategic planning efforts in the area of emerging lighting strategies.
EM&V Project Number 8. Energy Efficiency Potential
A bottom-up assessment of measure/end use savings and program participation levels is needed to inform a new EE potential study for the years 2013-2015 and beyond. This study will build as feasible on existing data and models utilized in the 2008 California Potential Study. New data collection/modeling will be gathered and utilized to ensure the accuracy of the inputs and projections. New methodologies will include a review of best practices and examination of potential as proposed through various existing building strategies in the California Strategic Plan.
With an updated EE Potential Study in hand, a new Goals Study as required in D. 08-07-047 will be undertaken. New methodologies will be examined, to allow the identification of goals based on assumptions of achievements of existing building and other targets as contained in the California Strategic Plan. Review will examine costs associated with these new strategies, and possible offsets to program costs with societal benefits, such as job creation.
EM&V Project Number 9. Ex-Ante Estimates Development
All ex ante measure parameters used to determine savings accomplishments and for future energy efficiency portfolio planning will come directly from the DEER database , which will include both DEER measures and "non-DEER" measure work papers. All ex-ante estimates are proposed to be updated by the end of 2010 for use in planning the portfolios that will be implemented in 2013. The ex-ante estimates will be developed using the best available data and methodologies.
The budget for this category includes (1) ex ante parameters updates, (2) statistical analysis for developing ex ante updates, (3) deemed measures cost studies, (4) customized project cost analysis, and (5) useful lives and technical degradation analysis.
EM&V Project Number 10. Data Management
The Energy Division proposes to continue its management and quality control of data, evaluation activities, and parameters used to calculate energy savings and cost-effectiveness. The budget for this category includes (1) a Data Quality and Data Management consultant contract, (2) updates and maintenance of energy efficiency websites (DEER, EEGA, CMS, CALMAC, etc.), (3) cost-effectiveness tool development, (4) avoided costs and GHG emissions updates, (5) data tracking and reporting system enhancements, and (6) Energy Division reporting.
EM&V Project Number 11. Best Practices and Methodology Improvements
Four studies are planned in this area, two being mandated by D.09-09-047.
· EM&V Technical and Institutional Framework. Ordering Paragraph 59 states that "Energy Division may hire a contractor to initiate in 2010 a comprehensive review of current Evaluation, Measurement & Verification technical and institutional frameworks." This is further described on pages 9 and 305 of the Decision: "The main purpose of this review will be to set a course to develop effective EM&V going forward, post-2012. However, to the extent this review will allow us to improve the 2010-2012 program cycle, we will do so."
· Behavioral Energy Savings Estimation Methods. Ordering Paragraph 60 requires 2010-2012 EM&V to undertake "consideration of methodologies to verify savings driven by behavior-based energy efficiency programs." This study will search for, review, identify, and develop as necessary solid methods for estimating the energy savings created by programs focused on changing energy user behavior.
· Methodology Development for Attribution Analysis. Closely related to behavioral energy savings measurement, improved methods for determining the attribution of energy savings are needed. The concepts of energy efficiency programs competing in forward capacity markets, of additionality in greenhouse gas emissions markets, and of free ridership and spillover in the energy efficiency programs arena need comparison, further analysis, and further development to meet the increasing needs for identifying causality in all these areas, and to identify what roles (if any) energy efficiency programs can play in the new markets.
· Improved Statistical Analysis Processes for Energy Efficiency Savings Estimation. By the end of this cycle, the utilities will have interval energy usage data for virtually all of their customers. With this vastly increased information about energy usage, statistical analysis of energy usage and other data becomes an increasingly powerful method of developing not only program energy savings estimates, but also ex ante estimates of measure savings. Savings estimates based on actual energy usage data have the advantage of incorporating effects of customer behavior in relation to installed measures. It is critical to prepare for the maximum effective use of this new data.
EM&V Project Number 12. Energy Consumption Surveys
The California Commercial End-Use Survey (CEUS) is a comprehensive study of commercial building sector end-use energy use. The survey captures detailed building systems data, building geometry, electricity and gas usage, thermal shell characteristics, equipment inventories, operating schedules, and other commercial building characteristics. Commercial premises are weighted and aggregated to building segment results. Available study results include floor stocks, fuel shares, electric and natural gas consumption, energy-use indices, energy intensities, and 16-day hourly end-use load profiles estimated for twelve common commercial building type categories.
The California Industrial End-Use Survey (IEUS) is a comprehensive study of industrial sector energy end-use energy. The mail, internet and on-site surveys and metering of some large process loads are expected to produce: Equipment saturations (including EE levels, vintages, and cogeneration), End use characteristics, Building characteristics, Space heating/cooling, Lighting, General production equipment, Industry specific process equipment, Energy Use (electric and gas) by INFORM (industrial forecasting model) end-use categories and industry groups and Load Profiles by utility area and industry group.
The California Lighting and Appliance Saturation Study (CLASS) database provides baseline information on residential appliance, equipment and lighting saturations and efficiencies. The overarching goal for the studies is to provide an accurate baseline in order to understand future energy savings potential and past accomplishments in the residential sector. The original study was completed in 2000 and repeated in 2005 to see what changes had taken place over the 5 year period. Repeating this study for a third time in 2010 will show the continuing effects of residential energy efficiency in California.
The Residential Market Share Tracking (RMST) study has monitored the market penetration of energy efficient appliances and lighting measures in California since 1999. RMST measures statewide and utility milestones for promoting short-term adoption of measures and long-term market acceptance of energy efficient technologies. In addition to the program implementers, beneficiaries of this research include federal and state agencies, regional and state energy efficiency organizations, trade organizations, equipment manufacturers, distributors, and retailers.
A total energy consumption evaluation pilot study will be conducted to assess the reduction in energy consumption resulting from the various energy efficiency programs and efforts in California. The value of individual energy efficiency efforts is uncertain without the measurement of performance of the whole system to link the efforts to actual reduction in energy consumption. Issues that arise from field measurements are that the actual energy performance of an energy efficiency measure does not align with the initial specification of the design intent. Some of the factors that contribute to these inconsistencies are the lack of system integration in design and operation, and the lack of training and work force necessary for the appropriate installation and maintenance of equipment.
Energy efficiency should be used in conjunction with performance metrics such as energy intensity in describing the mathematical relationship between energy use and service output. The intensity component, the energy use rate, is the ratio of the total consumption to a unit of measurement (e.g. Btu/square-foot-hour, million Btu/household, energy/gross output, energy/industrial production etc.). A decrease in energy intensity over time may correspond to an increase in energy efficiency depending on the level other structural and behavioral effects. A good measure of energy intensity should identify (or remove from a measure) as many of the behavioral and structural changes that affect the energy intensity (but are generally agreed upon to be unrelated to energy efficiency) as is computationally feasible within budget limitations and data availability.
The study will design and implement an EM&V approach for the assessment of energy consumption for the different end-use sectors in California including:
a. Defining energy intensity indicators for the different end-use sectors;
b. Identifying behavioral and structural factors that can affect energy intensity but not related to energy efficiency improvements.
c. Identifying the effects of the IOUs programs in the reduction of energy consumption for a given end-use sector;
EM&V Project Number 13. Portfolio Financial and Management Audits
The CPUC Utility Audit, Finance, and Compliance Branch (UAFCB) staff and ED staff will perform an evaluation of the IOU energy efficiency portfolio financial administration and management systems. A financial audit will consist of a review of the financial statements of each utility's energy efficiency operations to determine if the statements are accurate, complete, and consistent with Commission policy and standard accounting practices. The management audit will be a systematic assessment of each utility's management procedures and the effective use of resources in implementing the energy efficiency portfolios.
EM&V Project Number 14. ED Master Evaluation Contractor Team
Please refer to section 5f "Procurement of management and technical consulting services for ED."
EM&V Project Number 15. CPUC staffing funded by EM&V
Consistent with current practice, a small portion of the EM&V funding will be set aside to fund a portion of the Energy Division's energy efficiency staff positions.
(END OF ATTACHMENT 1)
Attachment 2
Process for Commission Oversight
of IOU EM&V Project Initiation
1. Project Formation: IOUs notify ED of their intention to conduct an EM&V project and solicit input from ED on the shaping of the project. ED may choose to waive this opportunity to participate if it chooses. The point of this step is to minimize potential delays in the following steps.
2. Project Description: Once the need for a project has been determined, the IOUs will prepare a project description (basically a high level scope of work, following reporting standards to be developed).
3. Project Tracking: The project description will be uploaded to the Energy Division's project tracking system.
4. Project Review and Approval: the project description will be available for review and approval by Energy Division for one calendar week.
4.1. ED will prioritize its review of projects and will reserve its review for projects of high importance (such as evaluations of strategic plan programs, "Big Bold Energy Efficiency Strategy" programs, and programs/measures with high forecasted savings) or projects that are clearly good candidates for coordination between IOUs and ED.
4.2. Within two weeks, ED will notify the IOUs if they intend hold the project to conduct a more detailed review and/or if ED requires more information on the project from the IOUs before approval can proceed. ED will provide the IOU with a letter of contractor rejection stating the specific conflict problem behind rejection of a proposed contractor, and provide an opportunity for discussion or conflict resolution before a final contractor decision is made. The IOU shall have two weeks from the date of receiving the letter of contractor rejection to discuss the issue, or file a "Motion for EM&V dispute resolution" or request for ADR. Otherwise, ED may finalize the selection of contractor.
4.3. If two weeks pass and ED has not already indicated that the project is approved or ED has not already held the project for further review, then the project will be considered approved and Energy Division's opportunity to review the project will be considered waived.
5. Project Initiation: Once the ED review and approval is completed (or waived) the IOUs may begin implementing the project in accordance with the project description.
6. Project RFP and Proposals: If the project requires competitive bidding, the IOUs will upload the RFP to the Energy Division project tracking system. If the project involves consultant proposals, the proposals will be uploaded to the Energy Division project tracking system.
7. Contractor Selection: If the project involves hiring a contractor, whether by competitive or directed bid, the IOUs will notify ED of their preferred contractor and other contractors who were considered and/or who submitted bids. ED will make the final selection of all EM&V contractors.
Process for Commission oversight of IOU EM&V project implementation
1. Project Reporting: The IOUS will upload project documents to the Energy Division project tracking system. The required project documents and standards for timing will be determined at a later time. Project documents will include EM&V work plans, schedules, methodologies, analyses, draft reports, and interim findings.
2. Project Briefings: The IOUs will provide briefings on all EM&V projects to ED at regular intervals.
3. Project Advisory Meetings: Certain projects will be selected by ED as requiring the opportunity for regular ED participation. For these projects, the ED liaison will be notified of project meetings.
(END OF ATTACHMENT 2)
Attachment 3 - Energy Division EM&V Questions and Recommendations From November 2009 ALJ Ruling
1. What are the respective roles of Energy Division and IOU EM&V staff for conducting EM&V projects?
i. Are the IOUs permitted to manage any impact evaluation or M&V projects that develop ex-ante savings estimates which may be used for determining portfolio performance, reporting accomplishments, or calculating incentives? If so, what are the Commission's expectations for rules and procedures for oversight of these projects?
Energy Division Recommendation - Question 1.i.
The IOUs should be permitted to manage projects to develop energy savings estimates in the specific case where there is no existing ex-ante estimate or the IOUs believe that an existing estimate is out of date and needs testing AND Energy Division is not already conducting or planning to conduct a project to develop estimates for the same measure. The IOUs should be required to seek approval from Energy Division before initiating such work and should proactively provide opportunities for Energy Division to review project milestones and provide input directly to the project manager. The Commission should clearly and explicitly authorize Energy Division to oversee such projects, including authorization to deny approval of projects that are not in the ratepayers interest. Energy Division's project approval will follow the process outlined in Energy Division's recommendations for questions 2,3, and 4.
ii. Is Energy Division expected and therefore permitted to initiate and manage evaluations that may be considered process or formative evaluations?
Energy Division Recommendation - Question 1.ii.
The Commission should clearly and explicitly authorize Energy Division to conduct any type of EM&V consistent with the following guidelines, which are adapted from the ED Straw Proposal Issued by ALJ Ruling on July, 7th, 2009 in A.08-07-021.
· Management of research projects that support the development of data, information, and tools needed to conduct regulatory oversight as well as to improve the Commission's energy efficiency policies. This may include the following types of research:
o Summative/ex-post impact evaluations.
o Evaluations and M&V conducted for the purpose of developing savings estimates.
o Evaluations and audits used to develop conclusions about program performance.
o Market studies required to inform Commission EE policies.
iii. Should ED have the authority to be involved in projects that develop ex-ante savings estimates, such as the non-DEER work papers, which are currently managed by the IOUs without any ED involvement?
Energy Division Recommendation - Question 1.iii.
The IOUs should be required to notify Energy Division of all workpaper development activities and should proactively provide opportunities for Energy Division to review methodologies and provide input to the workpaper authors. ED involvement at this stage will streamline the review of final workpapers and will ensure greater reliability of workpaper savings estimates. Energy Division's involvement in workpaper projects will follow the process outlined in Energy Division's recommendations for questions 2,3, and 4.
2. Should ED be responsible for approving IOU EM&V projects? Should there be exceptions to this process for expedited projects?
3. Current policy requires ED to approve all IOU EM&V contractors in order to manage contractor conflicts of interest. Should this process continue or be modified?
4. Should ED have the authority to be involved in IOU EM&V projects?
Energy Division Recommendation - Questions 2,3, and 4:
Energy Division proposes that its involvement in authorizing and reviewing IOU EM&V projects, including the ex-ante savings estimation projects discussed in 1.i. and 1.iii. above, be managed according to the following procedures, adapted from the ED Straw Proposal:
Process for Commission oversight of IOU EM&V project initiation
8. Project Formation: IOUs notify ED of their intention to conduct an EM&V project and solicit input from ED on the shaping of the project. ED may choose to waive this opportunity to participate if it chooses. The point of this step is to minimize potential delays in the following steps.
9. Project Description: Once the need for a project has been determined, the IOUs will prepare a project description (basically a high level scope of work, following reporting standards to be developed).
10. Project Tracking: The project description will be uploaded to the Energy Division's project tracking system.
11. Project Review and Approval: the project description will be available for review and approval by Energy Division for two calendar weeks.
11.1. ED will prioritize its review of projects and will reserve its review for projects of high importance (such as evaluations of strategic plan programs, BBEES programs, and programs/measures with high forecasted savings) or projects that are clearly good candidates for coordination between IOUs and ED.
11.2. Within two weeks, ED will notify the IOUs if they intend hold the project to conduct a more detailed review and/or if ED requires more information on the project from the IOUs before approval can proceed.
11.3. If two weeks passes and ED has not already indicated that the project is approved or ED has not already held the project for further review, then the project will be considered approved and Energy Division's opportunity to review the project will be considered waived.
12. Project Initiation: Once the ED review and approval is completed (or waived) the IOUs may begin implementing the project in accordance with the project description.
13. Project RFP and Proposals: If the project requires competitive bidding, the IOUs will upload the RFP to the Energy Division project tracking system. If the project involves consultant proposals, the proposals will be uploaded to the Energy Division project tracking system.
14. Contractor Selection: If the project involves hiring a contractor, whether by competitive or directed bid, the IOUs will notify ED of their preferred contractor and other contractors who were considered and/or who submitted bids. ED will make the final selection of all EM&V contractors.
Policy Issue #3 (part of C. in ED Straw Proposal)
Process for Commission oversight of IOU EM&V project implementation
4. Project Reporting: The IOUS will upload project documents to the Energy Division project tracking system. The required project documents and standards for timing will be determined at a later time. Project documents will include EM&V work plans, schedules, methodologies, analyses, draft reports, and interim findings.
5. Project Briefings: The IOUs will provide briefings on all EM&V projects to ED at regular intervals.
6. Project Advisory Meetings: Certain projects will be selected by ED as requiring the opportunity for regular ED participation. For these projects, the ED liaison will be notified of project meetings.
Finally, ED will exercise the authority granted to Commission staff under Public Utilities Code Section 314 (a), as needed, to review process evaluation plans and results.
5. Should ED have the authority to allocate the authorized EM&V budget between ED and IOU managed EM&V projects according to the overall EM&V priorities?
Energy Division Recommendation - Question 5:
The Commission should grant ED authority to approve IOU projects as discussed in the recommendation on questions 2,3, and 4 above. With this authority and the adoption of the prioritization process discussed in the Joint IOU/ED EM&V Plan, ED believes that a specific prior allocation to IOU managed projects above and beyond the 15% minimum to fund EM&V staff is unnecessary. Nevertheless, ED anticipates that the IOUs will request, and are likely to be granted responsibility to manage a sizable share of the EM&V work.
ED believes that the intention of the following statement on page 301 of Decision 09-09-047, "EM&V plans and budgets for 2010-2012 should be categorized in accordance with the first four objectives articulated above, and will be prioritized for approval in following with the most pressing needs across each category" is to allocate EM&V resources according to overall research priorities, rather than across organizations responsible for implementing EM&V projects.
6. How should major disputes arising out of the EM&V work be managed? When should these disputes be elevated to the full Commission for resolution?
Energy Division Recommendation - Question 6:
Energy Division maintains its recommendations articulated in the ED Straw Proposal, excerpts reproduced below:
Project-Specific EM&V Plans:
If parties continue to take issue with the final work plans, a party or parties may file a motion with the Assigned ALJ and provide evidence for why the plans should be changed and how. The ALJ will resolve the dispute and direct Energy Division and/or the IOUs to revise the plans accordingly via ruling.
EM&V Technical Reports
If parties continue to take issue with the final EM&V technical reports, a party or parties may file a motion with the Assigned ALJ and provide evidence for why the report is deficient and what changes to the report would be necessary to correct the deficiency. The ALJ will resolve the dispute and direct Energy Division and/or the IOUs, via ruling, to prepare an addendum to the report correcting the deficiency. The addendum will be posted on the same website where the draft reports are posted.
7. How extensively should IOUs be involved in ED EM&V projects?
Energy Division Recommendation - Question 7:
Energy Division maintains its recommendations articulated in Section C of the ED Straw Proposal "Stakeholder Input Process and Approval of EM&V Projects," as well as the informal interactions proposed in the Joint IOU/ED EM&V Plan.
8. What is the appropriate level of public involvement in EM&V projects? Should certain EM&V project be exempted from a full public process? How will the exempted EM&V projects be determined?
Energy Division Recommendation - Question 8:
In their comments on the ED Straw Proposal, the IOUs expressed concerns that engaging with the public on every EM&V project, as proposed in the ED Straw Proposal, would be ineffective and would slow down the implementation of time-sensitive projects. At least one utility has proposed that projects under a specific budget should be exempt from the type of public process proposed in Section C of the ED Straw Proposal and that the key stakeholders for process evaluations are limited to program administrators and implementers. ED believes that there will be IOU EM&V projects that will not require an intensive public vetting process, but we do not believe the project budget is a reasonable indicator of the need for public vetting. Additionally, ED strongly believes that ratepayers and the CPUC are in fact key stakeholders for process evaluations. To ensure that the appropriate EM&V projects are publically vetted and that time-sensitive projects are not delayed, ED recommends that the Commission grant ED authority to determine which EM&V projects should and should not undergo public vetting.
9. Should all IOU EM&V related projects, regardless of funding source (such as projects that develop savings estimates for non-DEER measures funded out of program funds), be required to follow the same policies and procedures that are required for EM&V funded projects?
Energy Division Recommendation - Question 9:
Energy Division recommends that the Commission clearly require, without exception, that all EM&V related projects, regardless of funding source, be required to adhere to the same policies and procedures as EM&V funded projects.
10. Should the IOUs modify program eligibility rules to require very large customized program participants to participate in evaluations if selected in a sample, as a condition for receiving EE funding?
Energy Division Recommendation - Question 10:
Many large project participants have either refused to participate in evaluations or have Energy Division believes it is reasonable to require participants who receive a large sum of EE funding and services to participate in evaluations, if needed. This participation would include on-site measurement and verification, as well as surveys of key participant personnel. Energy Division proposes to review past projects with the IOUs to determine the EE incentive threshold above which participation in evaluations would be obligatory.
Energy Division will endeavor to reduce the burden of participating in evaluations by coordinating with the IOU implementation and inspection process. Energy Division recommends that the Commission require the IOUs to cooperate with ED in this regard.
11. Should the Commission allow the IOUs the opportunity to count savings from behavior based programs?
i. How should the Commission develop EM&V methodologies to verify savings driven by behavior-based efficiency programs?
ii. What analytical issues are raised by changing policy to allow credit and require measurement of savings driven by behavior-based efficiency programs (i.e. savings persistence, potential double-counting of savings by other resource programs, potential double-counting of savings claimed as part of the conservation benefits assumed to underlie Advanced Meter Infrastructure (AMI) business cases [PG&E - D.09-03-026; SCE - D.08-09-039; SDG&E - 07-04-043])?
Energy Division Recommendation - Question 11:
Energy Division believes that the categories of behavior based programs need to be more specifically defined and measurement issues need to be clarified before categorically recommending savings credit from behavior based programs. Energy Division believes there are significant intersecting issues with the IOUs' AMI programs. For instance, we believe it is the intent of the AMI program to provide customers with usage data to help them manage their energy consumption through conservation. Comparative usage reporting and benchmarking could be provided as part of the bundle of AMI services and may thus be considered AMI generated conservation savings. ED will evaluate the IOUs comparative usage programs using experimental design consistent with SB 488. ED proposes to refine the reporting required of IOUs by SB 488 with IOU input. Measuring and quantifying savings from other behavior based programs, such as Marketing, Education, and Outreach may be complicated and developing protocols for this measurement approach may take some time.
Energy Division recommends that the Commission consider forming a working group, facilitated by Energy Division, to explore these issues.
(END OF ATTACHMENT 3)
1 The most recent energy savings goals were adopted in D.08-07-047.
2 The structure of shareholder incentives is currently under review in Rulemaking (R.)09-01-019. As stated in D.09-12-045 at 4 in that docket, "We continue to believe that prospectively, reforms to the existing mechanism should be pursued that reasonably produce meaningful incentives to achieve the Commission's energy efficiency goals through simplified approaches designed to avoid the level of controversy over detailed technical methodologies that have characterized the RRIM process to date." While possible outcomes of that proceeding could include de-linking shareholder incentives from EM&V study results, in this decision we will not prejudge any potential outcomes in R. 09-01-019. Therefore, we continue to assume the current structure whereby EM&V studies directly impact shareholder incentives.
3 It appears that PG&E's $49.5 million level is inclusive of the $18.75 million needed to maintain IOU staffing level.
4 D.05-11-011, Attachment 3, Table 1 ("IOU managed evaluation projects" line item).
5 As sponsors (along with ED) of the Joint Plan, there was no need for the IOUs to provide comments on this document.
6 To the extent that there are differences between the ED Straw Proposal and the combination of ED's positions in the Joint Plan and its recommended resolution of outstanding EM&V issues, we consider the latter views to supersede the Straw Proposal.
7 The McNulty Study was filed by SDG&E in an IOU Petition for Modification of D.07-09-043 and D.08-01-042, which effectively challenged ED's first Verification Report of 2006 and 2007 energy savings.
8 ED has e-mail documentation of a duplicate study by PG&E involving CFL interactive effects, a duplicative study by PG&E of evaluation of oil field efficiency measures, and an SCE process evaluation study on upstream lighting which partially duplicated an ED impact evaluation (without going through the process set out in D.05-01-055).
9 Ex-ante estimates are forecasted assumptions for energy savings and cost-effectiveness parameters, which are used to evaluate energy efficiency program proposals submitted by the utilities in their periodic energy efficiency portfolio applications. Ex-ante estimates are also used to report energy savings from energy efficiency measures prior to determining ex post, or actual, energy savings.
10 In the situation where a proposed contractor would simultaneously be engaged by both the IOU and Energy Division, this should not per se be considered a disqualifying conflict. Energy Division should consider whether limited availability of qualified contractors would override any such conflict.
11 A process evaluation is the systematic analysis of the development, design, and actual implementation of a strategy or program; an assessment of whether program activities were implemented as planned; and an assessment of whether expected outputs were actually produced.
12 "Workpapers" refers to documentation prepared by the program administrators or program implementers that documents the data, methodologies, and rationale used to develop ex-ante estimates that are not in already contained in the Database for Energy Efficiency Resources (DEER).
13 See July 7, 2009 ALJ Ruling, Attachment A, at 7-12.
14 Public Utilities Code Section 314 (a) provides that Commission staff may, at any time, inspect the accounts, books, papers, and documents of a public utility.
15 ED's November 2008 draft first Verification Report recommended rewards and penalties of under $10 million for each of the four IOUs. Before the first Verification Report was completed, the IOUs filed a Petition for Modification of D.07-09-043 and D.08-01-042 arguing, among other things, that there were fundamental flaws in the underlying data and analysis used in the ED's verification process for 2006 and 2007 energy savings. The IOUs collectively sought shareholder incentive rewards for energy savings in the amount of over $150 million. Ultimately, the Commission in D.08-12-059 determined that the IOUs should be awarded an interim amount of $82 million, with a 65% holdback of claims for future review. D.09-12-045 awarded the IOUs $61 million of their subsequent claim, with 35% holdback for further review.
16 There have also been ED proposals issued for informal comment.
17 Unless ED is a party, any response to a formal filing would not be part of the record unless placed in the record by Ruling or other Commission action. It is not clear whether ED could be made a party to a proceeding. In any case, ED has never sought to file such a reply or become a party to an energy efficiency proceeding.
18 To the extent that an EM&V Motion would be delayed past the deadlines required by this Decision in order to include a statement from ED, the filing party should ask the assigned ALJ for leave to late-file the EM&V Motion.
19 "Calculated savings programs" refers to programs that offer rebates for custom projects (typically for large commercial and industrial customers), for which the rebate and ex-ante energy savings are calculated for each individual project. "Deemed savings programs" refers to programs that offer predetermined rebates and assume a predetermined energy savings for a range of conditions (i.e. climate, building type, etc.) for a predetermined set of energy efficiency products and services. Ex-ante deemed rebates and savings assumptions are not adjusted for each individual project.
20 SB 488 was approved by the Governor on October 11, 2009. The statute requires IOUs that have comparative energy usage disclosure programs to report program energy savings to the Commission and requires the Commission to use experimental design evaluations to determine net energy savings from these programs and report to the Energy Commission and the Legislature the evaluation results and any action undertaken by the Commission in response to the evaluation.
21 The California Evaluation Protocols provide standard procedures and methodological choices for conducting various types of evaluations.
ftp.cpuc.ca.gov/puc/energy/electric/energy+efficiency/em+and+v/EvaluatorsProtocols_Final_AdoptedviaRuling_06‐19‐2006.doc.
22 SDG&E/SoCalGas allocated 75% to ED and 25% to the utilities in Advice Letters 2034-E/1809 G and 3912, respectively. This decision supercedes those Advice Letters.
23 Throughout this document "EM&V," if not otherwise indicated, is understood to refer to both traditional Evaluation, Measurement and Verification work as well as the Energy Division's policy and planning activities. (See Decision 09-09-047 pages 295-298 and 387 for more on policy and Planning as included in the EM&V budget and activities).
24 http://www.eval.org/publications/guidingprinciples.asp
25 http://www.nspe.org/Ethics/CodeofEthics/index.html
26 http://www.ashrae.org/publications/detail/16451
