8. Counting Savings from
Behavior Based Programs

In D.09-09-047 we directed an investigation into the feasibility of crediting savings from behavior-based energy efficiency programs. In following with this directive, Question 10 of the November 20 Ruling asked: "Should the Commission allow the IOUs the opportunity to count savings from behavior based programs? How should the Commission develop EM&V methodologies to verify savings driven by behavior-based efficiency programs? What analytical issues are raised by changing policy to allow credit and require measurement of savings driven by behavior-based efficiency programs (i.e. savings persistence, potential double-counting of savings by other resource programs, potential double-counting of savings claimed as part of the conservation benefits assumed to underlie Advanced Meter Infrastructure (AMI) business cases [PG&E D.09-03-026; SCE D.08-09-039; SDG&E D.07-04-043])?"

ED believes that the categories of behavior-based programs must be well-defined and measurement issues clarified before categorically recommending savings credit from behavior based programs, stating that if defined too broadly, evaluation resources could become unduly tied up in measuring savings from which represent a small fraction of overall portfolio savings. In addition, ED believes there are significant intersecting issues with the IOUs' AMI programs. For instance, ED believes it is the intent of the AMI program to provide customers with usage data to help them manage their energy consumption through conservation. Comparative usage reporting and benchmarking could be provided as part of the bundle of AMI services. Thus, crediting these savings in the context of energy efficiency programs will require careful accounting to ensure that they have not been either counted, or paid for, twice. ED recommends that the Commission consider forming a working group, facilitated by ED, to explore these issues.

For the purpose of this Decision, we will restrict the definition of behavior based programs to the "comparative energy usage disclosure programs" defined in SB 488. As defined by SB 488, comparative usage programs are specifically programs "...pursuant to which an electrical corporation or gas corporation discloses information to residential subscribers relative to the amount of energy used by the metered residence compared to similar residences in the subscriber's geographical area." We understand that certain non-residential market sectors may offer opportunities for comparable or greater savings under similar programs, and thus do not restrict our definition to residential applications.

PG&E points out that Senate Bill (SB) 488²⁰ encourages the pursuit of comparative home energy reporting, a specific type of behavior based program, which should be evaluated using experimental design approaches. PG&E and OPower suggest that the use of experimental design can help prevent double-counting from behavior based programs and other initiatives, such as AMI. OPower provides two examples of completed experimental studies of their program as well as a proposed protocol, which PG&E and Sempra recommend be added to our evaluation protocols. PG&E, SCE, and OPower point out that experimental design is already a method accepted by the Commission and is included in the California Evaluation Protocols.²¹

DRA recommends that ED develop a methodology to isolate behavior based savings from EE resource programs, non-EE programs, AMI influences, as well as energy efficiency messages from sources outside of the utilities. DRA is concerned that measurement of behavior based savings without improved methods will be daunting, costly, and controversial; and questions whether such methods can be implemented objectively. TURN believes that extensive research is still needed before the Commission should commit to allowing the counting of savings from behavior based programs. PG&E believes that the Commission must act quickly with existing methods and not wait to develop any new methods or protocols.

SCE requests that the Commission to create a regulatory environment that encourages behavior based programs, emphasizing that behavior change and conservation are critical to achieving market transformation and reductions in energy use. SCE recommends a performance incentive mechanism that tracks and credits behavior and conservation goals, including the measurement of energy savings from behavior based programs. OPower states that the IOUs have indicated that they want to operate behavior-based efficiency programs at scale, and that they are waiting for this Commission's approval before they do, and in at least in one case, will only proceed with such a plan if the Commission recognizes behavior as an efficiency resource. SCE argues that the greatest uncertainty with measuring behavior based programs is savings persistence. SCE therefore suggests that savings from behavior based programs be estimated with shorter ex-ante effective useful lives while ED and the utilities identify and develop more reliable methods for estimating energy savings created by programs focused on changing energy user behavior. NRDC suggests that behavior based programs be assumed to provide savings only during the period that the program is in place.

SDG&E/SoCalGas supports the estimation of savings from behavior based programs, so long as the savings are reliable and verifiable. SDG&E/SoCalGas advises that the full value of benefits resulting from California's Smart Meter (also known as advanced metering infrastructure or AMI) investment must be recognized and quantified to encourage effective utility programs.

In D.09-09-043, we indicated our intent to consider EM&V methodologies to account for behavior based programs. We reasoned that because many of the programmatic directives in that decision marked a shift towards market transformation consistent with Strategic Plan objectives, the Commission should consider ways to credit savings from new programmatic approaches focused on generating measured energy use reductions through behavioral motivation.

We agree with the utilities that it is within our energy efficiency program's best interest to create a regulatory environment that encourages behavior change and conservation. Such a regulatory environment necessarily includes the measurement and crediting of energy savings from programs that focus on behavioral motivations to generate measurable savings. We also understand ED's concern that if defined too broadly, this pursuit could consume many resources and distract away from other priority EM&V activities. As DRA warns, we must be cautious not to commit to an overly complex, costly, and controversial measurement system. All parties recognize the need to avoid double-counting, with which we agree.

Given the parties' comments and testimony submitted on this issue, we are persuaded that it is reasonable to measure savings from certain behavior based programs. To the extent that any program holds potential to produce significant verifiable savings, it is appropriate to attempt to estimate such savings, particularly when consistent with the overall policy direction we have adopted for energy efficiency.

We thus adopt a policy to measure and count savings from comparative usage programs, as defined in this Decision, using experimental design methodologies contained within the California Evaluation Protocols. We defer to the prioritization process described in the EM&V Plan adopted in this decision to make decisions regarding which behavior based programs will be evaluated and specifically how those programs will be evaluated.

These programs have intersections with several other categories of program activities already underway, such as AMI. As such we must take special care to ensure that savings credited to these programs do not represent double-counting. The experimental design method, as described in the California Evaluation Protocols, and spelled out in greater detail in parties' comments, is well equipped to deal with most of the analytical issues raised by the overlap of the savings targeted by comparative energy use reports, and programs already under way through Commission directive. So long as the evaluation is set up to compare two populations which in statistical terms are in no way different except for the treatment of the program, the measured savings should be those attributable to the program, provided the experiment is properly designed.

As shown in the record, deployments of comparative energy use programs have yielded savings on the order of 1.5 to 3.5 percent across sample populations. However, due to overlap with other initiatives targeting behavior change, measurable savings from service areas within our jurisdiction may differ, lending uncertainty to any savings we may project for these programs. We thus, propose a slightly different process for the crediting of savings from these programs for 2010-2012. Utilities will not be allowed to submit workpapers for ex ante numbers which draw from other sources to project savings for these programs. Our policy determination in D.09-09-047 to freeze ex ante values could potentially lock in overly optimistic savings projections for this novel brand of resource program. Instead we commit only to crediting ex post savings for behavior programs in the 2010-2012 program cycle. As such, the onus is on the program provider to make the case to the utility that the program provides added value to efforts already underway, and that projected savings will materialize as real and verifiable.

²⁰ SB 488 was approved by the Governor on October 11, 2009. The statute requires IOUs that have comparative energy usage disclosure programs to report program energy savings to the Commission and requires the Commission to use experimental design evaluations to determine net energy savings from these programs and report to the Energy Commission and the Legislature the evaluation results and any action undertaken by the Commission in response to the evaluation.

²¹ The California Evaluation Protocols provide standard procedures and methodological choices for conducting various types of evaluations.

ftp.cpuc.ca.gov/puc/energy/electric/energy+efficiency/em+and+v/EvaluatorsProtocols_Final_AdoptedviaRuling_06‐19‐2006.doc.