2. Background
2.1. The EM&V Policy Framework
In 2003, the Commission, in collaboration with the California Energy Commission (CEC) and the now defunct California Consumer Power and Conservation Financing Authority developed California's first Energy Action Plan (EAP).2 The EAP sets forth a loading order that prioritizes which energy resources California will use to meet its future energy needs. The loading order stipulates that energy efficiency is California's "resource of first choice." Since the loading order issued, the Commission has invested in energy efficiency programs designed to displace or defer costly supply-side alternatives.
It is in the context of energy efficiency as a resource that the Commission's existing EM&V policy framework took shape. Decision (D.) 05-01-055 returned California's Investor Owned Utilities (IOUs)3 to the role of energy efficiency program administrators and tasked the Commission's Energy Division with EM&V of the utility programs. D.05-01-055 defined the objectives of EM&V as follows:
1) measure and verify energy and peak load savings for individual programs, groups of programs, and at the portfolio level; 2) generate data for savings estimates and cost-effectiveness inputs; 3) measure and evaluate the achievements of energy efficiency programs, groups of programs and/or the portfolio in terms of the "performance basis" established under Commission-adopted EM&V protocols; and 4) evaluate whether programs or portfolio goals are met. (D.05-01-055, at 12.)
D.05-04-051 subsequently adopted policy rules for energy efficiency and defined the utilities' "performance earning basis."4 Informed by these fundamental EM&V components and plans, the IOUs designed their 2006-2008 energy efficiency portfolios. The portfolios, subsequently approved in D.05-09-043, were largely made up of up-stream lighting programs, that primarily focused on the Compact Fluorescent Lighting (CFL) markets.
In 2007 the Commission issued D.07-09-043, laying the groundwork for a Risk/Reward Incentive Mechanism (RRIM) which allows utility shareholders to profit by achieving defined energy savings targets and sets up penalties for significant underperformance. Determination of utility earnings or penalties through the RRIM was to rest on evaluations of program performance. The need to inform these determinations added new and greater emphasis on the transparency, accuracy, and reliability of EM&V results. Since its inception, the mechanics of the RRIM, as well as the defined targets (goals), have been highly contentious. Parties continue to disagree over whether the RRIM provides effective incentives to foster improvements in energy efficiency program design or performance.
In 2008 two significant developments reshaped California's energy efficiency landscape and added new objectives to those identified in the EAP. First, pursuant to Assembly Bill (AB) 32 (California's Global Warming Solutions Act), the California Air Resources Board (CARB) expects to achieve up to 15% of the mandated reductions in Greenhouse Gas (GHG) emissions through energy efficiency. Thus, with the passage of AB 32, energy efficiency became not only the state's energy resource of choice, but also a primary factor in achieving California's GHG reduction targets.
The second development, inspired in part by AB 32, was the development of the California Long Term Energy Efficiency Strategic Plan (Strategic Plan).5 The Strategic Plan envisions an energy efficient future for each customer segment and identifies market transformation strategies to help transform utility energy efficiency programs. The Strategic Plan also directs IOU energy efficiency programs to transition away from measures which provide short-term energy savings (i.e., CFLs) in favor of more comprehensive, long-term savings.
In D.08-07-047 the Commission updated its energy efficiency goals. Prior to D.08-07-047 energy efficiency goals were limited to energy savings achieved by IOU programs. This had the unintended outcome of creating disincentives for cooperative programs.6 D.08-07-047 replaced the earlier, narrower, definition of goals with "Total Market Gross" goals. Total Market Gross goals reflect the Commission's expectation that utility programs should complement and enhance state building standards, expected federal appliance standards, Big Bold Energy Efficiency Strategies, and AB 1109.7
In 2009 the Strategic Plan's emphasis on market transformation and long-term savings began to be incorporated into the IOUs' programmatic energy efficiency activities. D.09-09-047 approved the IOUs' 2010-2012 portfolios and began implementing energy efficiency programs designed to achieve the objectives of AB 32 and the Strategic Plan. In D.10-04-029 the Commission authorized a Joint Energy Division/IOU EM&V plan to evaluate the 2010-2012 programs. The evaluation of the 2010-2012 programs will help bridge the gap between the past and future of energy efficiency. Evaluations will measure savings from behavior-based programs, progress toward the market transformation objectives outlined in the Strategic Plan, and quantify the demand side energy resources created as a result of portfolio investments.
California is now being served by a multitude of energy efficiency programs. In addition to the Commission's energy efficiency programs, energy efficiency services are being provided through the American Recovery and Reinvestment Act (ARRA), local governments and private entities, and building and appliance standards. Each of these services is provided by an independent administrator with its own funding mechanism, program structure, and performance metric. This presents new challenges for EM&V.8 Among other things, determinations of program impact and cost-effectiveness will have to meld multiple expenditure streams. The success of these programs will depend on our ability to integrate efforts and leverage resources just as disputes over who gets to claim energy efficiency savings (attribution) will inhibit success. This raises new challenges for how we go about assessing the effectiveness and merits of ratepayer expenditures to support utility energy efficiency programs.
In short, the policy framework underlying energy efficiency has undergone significant transformation since its conception more than five years ago. Energy efficiency faces new priorities and challenges in an evolving market. The time is ripe to take stock of the current framework to ensure that it meets California's needs going forward.
2.2. The EM&V Methodological Framework
Two documents contain the Commission's methods and best practices to date: the California Evaluation Framework (Evaluation Framework)9 and the California Energy Efficiency Protocols (Protocols).10 The Evaluation Framework was developed though the collaborative work of the IOUs, Energy Division and TecMarket Works, a team of professional evaluators offered recommendations for consistent methods and best practices for a wide range of evaluation questions outlined options for a cyclical approach to planning and conducting evaluations of energy efficiency programs. The Protocols were initially adopted by Administrative Law Judge (ALJ) ruling in April of 2006 as a follow up to the Evaluation Framework and were offered as a more prescriptive guide for conducting evaluation and allocating resources. Minor updates were adopted by ruling in January 2007.11 The Protocols were developed by TecMarket Works specifically to guide evaluation of the 2006-2008 IOU energy efficiency program cycles. The Protocols specify in detail acceptable approaches and procedures for the evaluation of IOU energy efficiency portfolios. The content of these documents has remained largely unchanged since 2006.
2.3. The 2006-2008 Energy Efficiency
Evaluation Report
On July 8, 2010, Energy Division issued a Energy Efficiency Evaluation Report for program years 2006-2008 (06-08 Evaluation Report).12 In the 06-08 Evaluation Report the previously discussed policy and methodological frameworks are combined to measure and verify energy savings, test the cost-effectiveness of IOU portfolios, and evaluate whether energy savings program goals were achieved. The completion of this energy efficiency EM&V effort is a remarkable accomplishment as it is the largest energy efficiency EM&V effort ever undertaken.
The 06-08 Evaluation Report finds that between 2006 and 2008, IOU programs saved 4,093 gigawatt-hours and 44 million therms, and reduced peak electric load by 779 megawatts. The number of tons of carbon dioxide reduced, 2.6 million, is also significant. Overall, the 2006-2008 portfolios were found to be cost-effective. The 06-08 Evaluation Report also includes recommendations for improving future EM&V. One recommendation speaks to changes that have occurred in California's energy efficiency policy framework and implications for future EM&V:
The Commission should consider evaluation priorities for future program cycles that recognize expanded program and policy objectives for energy efficiency. The evaluation framework for 2006-2008 may not address the multiple and diverse evaluation needs for meeting AB32, the California Strategic Plan for Energy Efficiency, and Long-Term Procurement Plan objectives.13
This recommendation is being implemented through D.10-04-029 and the execution of EM&V for the 2010-2012 program cycle and will continue through the 2013-2015 cycle. The challenges and opportunities posed by this recommendation are central to the EM&V review addressed in this decision.
2.4. EM&V Review Procedural Background
D.09-09-047 identified the need for a comprehensive review of the Commission's existing energy efficiency EM&V practices. The Commission explained that the purpose of the review was to "set a course to develop effective EM&V going forward, post-2012."14 On November 20, 2009 the Commission approved an Order Initiating Rulemaking (OIR), that initiated Rulemaking (R.) 09-11-014. The OIR included within the rulemaking a "review and streamlining of our EM&V protocols and processes."15 On March 18, 2010 the Commission held a Pre-Hearing Conference (PHC) to begin considering several issues within R.09-11-014, including the comprehensive EM&V review ordered by D.09-09-047. The Division of Ratepayer Advocates (DRA), Marin Energy Authority (MEA), PG&E, SCE, SoCalGas and SDG&E (Joint Parties), The Utility Reform Network (TURN), and Women's Energy Matters (WEM) filed PHC Statements. The same parties, as well as the City and County of San Francisco (CCSF) and Natural Resources Defense Council (NRDC), filed Reply PHC Statements.
On May 21, 2010 the Assigned Commissioner issued a Ruling and Scoping Memo (May 21st ACR) that continued our EM&V review. The ruling provided background on the Commission's existing EM&V methodological and policy framework, described in detail the impetus for the review, outlined the process by which the review would be conducted, identified the issues within scope, and called for party input.16 Through comments and reply comments, input was received from CCSF, DRA, PG&E, SCE, Joint Parties, TURN, WEM, and OPOWER.17 A second ACR seeking additional party input was issued on July 2, 2010 (July 2nd ACR). In addition to the parties above, the CEC and Efficiency 2.0 filed comments and reply comments responding to the July 2nd ACR.
The May 21st ACR and July 2nd ACR sought party input on six key questions, which can be paraphrased as follows:
a. Should the Commission's EM&V objectives, as defined in D.09-09-047 be amended and if so, how?
b. Should the Commission's established EM&V Protocols and Evaluation Framework be amended or expanded and, if so, how?18
c. What are the merits of and challenges associated with:
d. What should California learn from other regions and states, including successful models for collaborative forums, to enable more effective EM&V?
e. What technological innovations may be brought to bear to support more effective EM&V?
f. How can the Commission's EM&V efforts better support related needs of other state agencies, including the California Air Resources Board (CARB) and the CEC, as well as Publicly Owned Utilities (POUs)?
2 "Energy Action Plan 1," California Energy Commission, California Public Utilities Commission and California Consumer Power and Conservation Financing Authority. May 8, 2003. Available at: http://docs.cpuc.ca.gov/word_pdfREPORT/28715.pdf.
3 In this Decision, "utilities" and "IOUs" refer to Pacific Gas and Electric Company (PG&E), Southern California Edison Company (SCE), San Diego Gas & Electric Company (SDG&E), and Southern California Gas Company (SoCalGas).
4 Energy Efficiency Policy Manual, Version 3.
5 www.californiaenergyefficiency.com.
6 D.04-09-060, Table 1A-1E.
7 AB 1109, the California Lighting Efficiency and Toxics Reduction Act, requires reductions in energy usage for lighting and encourages the use of more efficient lighting technologies. ( http://www.leginfo.ca.gov/pub/07-08/bill/asm/ab_1101-1150/ab_1109_bill_20071012_chaptered.pdf)
8 "Lessons Learned and Next Steps in Energy Efficiency Measurement and Attribution: Energy Savings, Net to Gross, Non-energy Benefits, and Persistence of Energy Efficiency Behavior." Skumatz, Lisa, Ph.D. and Skumatz Economic Research Associates (SERA). November 2009.
9 ftp://ftp.cpuc.ca.gov/Egy_Efficiency/CaliforniaEvaluationFrameworkSept2004.doc.
10 ftp://ftp.cpuc.ca.gov/puc/energy/electric/energy+efficiency/em+and+v/EvaluatorsProtocols_Final_AdoptedviaRuling_06-19-2006.doc.
11 http://docs.cpuc.ca.gov/efile/RULINGS/63294.pdf.
12 http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/EM+and+V/2006-2008+Energy+Efficiency+Evaluation+Report.htm.
13 06-08 Evaluation Report at 125.
14 D.09-09-047, at 302.
15 Order Initiating Rulemaking, November 20, 2009, at 3.
16 Assigned Commissioner's Ruling and Scoping Memo, Phase I. May 21, 2010.
17 Comments and Reply Comments were filed on June 4 and 18, 2010, respectively.
18 See California Energy Efficiency Evaluation Protocols
(ftp://ftp.cpuc.ca.gov/puc/energy/electric/energy+efficiency/em+and+v/Evaluators Protocols_Final_AdoptedviaRuling_06-19-2006.doc) and The California Energy Efficiency Evaluation Framework (ftp://ftp.cpuc.ca.gov/Egy_Efficiency/CaliforniaEvaluationFrameworkSept2004.doc).
19 Macro Consumption Metrics are tools of evaluation that use econometric models to assess the aggregate impact of energy efficiency policy on energy consumption. These metrics are distinguished from other methods of impact evaluation because they do not rely on the sum of a series of more granular studies.
20 Experimental Design is a research method used to determine net energy savings by comparing the energy consumption of treatment and non-treatment (control) groups.
21 D.09-09-047 defines Market Transformation as "long-lasting, sustainable changes in the structure or functioning of a market achieved by reducing barriers to the adoption of energy efficiency measures to the point where continuation of the same publicly-funded intervention is no longer appropriate in that specific market. Market transformation includes promoting one set of efficient technologies, processes or building design approaches until they are adopted into codes and standards (or otherwise substantially adopted by the market), while also moving forward to bring the next generation of even more efficient technologies, processes or design solutions to the market." Market Transformation Metrics are measures of the change in the structure or functioning of a market for energy efficiency products or services caused by a specific market intervention.
