17. Other Portfolio Direction
17.1. Ex Ante Review and Updates
Ex ante savings estimates are the foundation for portfolio planning and reporting accomplishments, and the starting point for evaluation and verification. Three concepts will guide our direction in this section:
· Use of best available information;
· Standardizing the process of freezing ex ante values for measures that can be frozen prior to start of a cycle; and
· Developing a clear, efficient process for freezing ex ante values for measures whose parameters cannot be frozen prior to the start of the cycle (primarily custom projects and non-DEER workpapers submitted mid-cycle).
The importance of DEER to all ex ante values and the wide range of information contained in DEER make the availability of this information necessary to ensure that parties are able to adequately review and comment on Staff's recommendations for DEER updates. As time permits, we expect Commission Staff to inform parties of its plans and progress on DEER updates and provide parties with information on changes to assumptions and expected values in advance of the release of its draft DEER update recommendations.
The IOUs, TURN, and NRDC all comment on what they characterize as the contentious nature of the DEER process. TURN argues that the main reason for this contentiousness is that DEER values impact incentives paid to IOU shareholders, and therefore utilities have a strong motivation to contest the values. TURN also asserts that, when ex ante values are frozen for use farther into the future, differences between ex ante and actual accomplishments increase, as will contention over the "to-be-frozen values."428 NRDC notes that disputes continue on the merits of evaluations used for inputs for the updates and on the reasonableness of DEER values compared to the rest of the country.429 PG&E comments that added complexity and related reduced transparency (due to a heavy reliance on derived values rather than evaluation results) have increased the contentiousness of planning cycles.430 SCE agrees with NRDC and PG&E and states that the "DEER process has become convoluted, which naturally decreases transparency and increases contentiousness."431
Some parties comment on the manner in which contentious issues concerning the DEER update should be resolved. PG&E recommends that the Commission direct the IOUs and Commission Staff to jointly identify whether any significant disagreements exist concerning updated DEER values and, if so, to jointly convene a working session with a third-party consultant, who has not participated in the ex ante updates, to attempt to reach consensus. If no agreement is reached, PG&E recommends that the Commission accept the third-party's recommendation and incorporate that recommendation into DEER.432 NRDC takes the position that the Commission should first focus on updating values that are agreed upon or have minimal controversy and should improve estimates for highly contested savings that affect a large portion of the portfolio."433 Synergy Companies recommends that the DEER update should use the most accurate ex ante values that are possible through agreement and due diligence by all parties. Synergy Companies points to the Regional Technical Forum (Northwest) as an example of good practices.434
DRA believes that ex ante assumptions should be developed through an independent process, and therefore disagrees with PG&E's proposal to require consensus building workshops with a third-party consultant. DRA comments that the workshop process would not lessen the complexity of energy efficiency, but rather would likely result in delays and contention in the Evaluation process.435 DRA disagrees with NRDC's suggestion that the Commission should focus on updating less controversial values. As an example, DRA points to the Net-to-Gross ratio, which is one of the most contentious metrics, in observing that the least controversial route is not necessarily the best route for improving portfolio program savings.436 TURN contends that the Commission never intended for the evaluation process to entail negotiations between Commission Staff and the IOUs.437
Parties ask that the Commission provide guidance for freezing the DEER data set for the duration of the 2013-2014 period.438 Parties also request that a cut-off date be established with respect to the data used for the update.439 Ecology Action recommends that savings values based on DEER and Commission-accepted workpaper values (as of January 1, 2013) should be frozen and applied to the full transition period, while updates to DEER and workpaper values that occur after January 1, 2013 should apply to the next program cycle.440 Parties assert that using frozen DEER values enables the IOUs to use the same fixed values for planning, implementing, and reporting.441 NRDC agrees with Ecology Action that the Commission should provide policy guidance to avoid applying assumptions retroactively.442 Synergy Companies opposes retroactive application of updated values.443 The Efficiency Council suggests that the Commission establish guidance for freezing the best available ex ante data set in the near-term for planning for the 2013-2014 period so that the parties are working from the same set of information and the planning and implementation process is not delayed while waiting for new data. It recommends continuing to study and update data throughout the transition period in order to establish a subsequent data set that can be similarly frozen for planning and implementing the post-2014 cycle.444 TURN opposes using frozen ex ante values through the portfolio cycle; instead the ex ante freeze should only be employed to prepare or plan for the bridge year portfolios.445 Parties also emphasize the need to adhere to a strict schedule and adopt updated values in a timely manner.446
Parties offer different recommendations for resolving the current contentious nature of the DEER process. TURN recommends that the evaluation and DEER update[s] should be detached from the shareholder incentive process and refocused on optimizing program design.447 TURN suggests a return to ex-post determination of accomplishments, but offers a possible solution of freezing ex ante data for planning purposes only, and then update ex ante values annually. PG&E recommends a transition to a "gross savings measurement methodology" that moves away from relying on Net-To-Gross ratio values.448 SCE agrees that DEER and evaluation processes should be detached from shareholder incentives, but disagrees with TURN's recommendation to update any frozen ex ante values during a program cycle.449 NRDC and the utilities agree that only "agreed upon" ex ante values should be frozen in this round of DEER. NRDC recommends that only "noncontroversial" ex ante values be adopted into DEER and urges the Commission to "set up a process ... to address the key unresolved disputes before planning for the post bridge period."450 PG&E and SCE agree comments that only "agreed-upon" 2006-2008 Evaluation values should be frozen "for use in portfolio planning and reporting."451
Several parties are concerned that DEER has not been developed in a transparent manner or with sufficient input and collaboration from stakeholders. These parties complain that they are not provided an opportunity to review DEER until after the work has been completed. PG&E states that the evolution of the development of DEER savings estimates into a process based heavily on energy simulations has caused a decrease in the transparency.452
PG&E and NRDC recommend a process similar to the Regional Technical Forum utilized in the Pacific Northwest as a method for simplifying the development of ex ante values. PG&E states that the Regional Technical Forum "is transparent, lacks contention among its stakeholders and has a proven track record." PG&E also suggests that following the Regional Technical Forum approach of using voluntary participation and support could significantly reduce costs to ratepayers for developing, maintaining, and administering ex ante values.453
We share parties' concerns about the controversies that surround updates to energy savings and cost-effectiveness parameter values. We recognize that most values for DEER and non-DEER measures include underlying complexity in analysis methodology and require interpretation in the use of data that can come from evaluation studies as well as other related research activities. We expect disagreement regarding specific values based upon differences in professional judgment. However, the Commission cannot adjudicate every disagreement about the values contained in the ex ante data. For this reason the Commission has given our Staff the responsibility of performing the review and making recommendations as to the values we should adopt. When we assigned our Staff the responsibility of maintaining and updating DEER together with other activities designated as "Research and Analysis in Support of Policy Oversight,"454 we stated:
We place these activities under the management of regulatory staff because they involve judgments that can influence either the development of performance targets or the measurement of program achievements. For example, in both DEER and Net-to-Gross ratio work, judgments need to be made about what specific energy savings numbers from which studies will be used to estimate energy savings for specific measures. Due to the conflict-of-interest concerns discussed above, the IOU Portfolio Managers would not be the appropriate entities to manage or directly contract for this type of work."455
We provide our Staff significant latitude in performing DEER and other policy oversight functions and do not require Staff to utilize any advisory groups to perform this work. As noted when this decision was made:
We decline, however, to involve one or more policy advisory groups in this area of responsibility on a standing basis, as some parties propose. We find this approach to be far more structured and potentially cumbersome than we believe is necessary. In performing the Research and Analysis functions, Commission and CEC staff should have full flexibility to obtain input from various sources, including working groups of experts or hired consultants, as they deem appropriate to the circumstances."456
In D.10-04-029 we set forth an approach for collaboration and dispute resolution between IOUs and Commission Staff for Evaluation studies. We affirm this recommendation below for evaluation activities in general and identify specific steps to ensure transparency and sufficient opportunity for party input in future DEER updates. However, we clarify here that the collaborative approach and dispute resolution process articulated in D.10-04-029 do not apply to the DEER update process. Recent experience suggests that such a process would interfere with our ability to regularly update ex ante values with the best available information. While we weigh the evidence and opinions of all parties in adopting ex ante values, we typically place a high weight on the recommendations of Commission Staff.
As TURN suggests, the Commission did not envision the ex ante update process, for either DEER or non-DEER values, to be a negotiation between Commission Staff and the utilities or other parties. We require that Staff seek input and review from parties on all ex ante values. However, Commission Staff should recommend ex ante values that reflect the best estimate of expected real portfolio accomplishments based upon the most appropriate and accurate data available.
We disagree with comments that DEER should be based only upon evaluation methods and results. As stated above, DEER falls under Commission Staff's broad responsibilities to undertake research and analysis in support of policy oversight.457 To perform these research and analysis functions we have given Commission Staff the flexibility to obtain input and perform research as it deems appropriate.458 Ex ante values used for planning must be the best estimates of the likely accomplishments of the utilities' proposed portfolios. We recognize that many of these planning values may be projections based on past experience, evaluations of past similar activities, and results of other related research.
We generally agree with parties' request that ex ante values should be adopted and held constant throughout the portfolio cycle. However, mid-cycle updates of ex ante values are warranted if newly adopted codes or standards take effect during the cycle.459 We anticipate that a new version of Title 24 will become effective January 1, 2014, and the specifics of changes to be made public in late 2012.460 The utilities shall make appropriate adjustments to their participation and incentive calculation rules as well as update their ex ante value calculations in response to codes and standards changes. Codes and standards changes shall, as discussed below, be reflected in both DEER and non-DEER ex ante values used for reporting of utility portfolio accomplishments.
In addition to the need to update DEER for mid-cycle significant codes and standards changes, DEER will require updates for use in 2015 and beyond planning activities. By the end of 2012, codes and standards changes that will be effective by 2014 should be known and thus content for mid-cycle DEER and non-DEER revision can be fully planned. In addition, Commission Staff's 2010-2012 evaluation activities that will provide results that can be used to inform 2015 and beyond planning will likely also be available. Commission Staff shall prepare and release a plan for DEER updates that covers the anticipated mid-cycle codes and standard changes as well as DEER updates for 2015 and beyond planning.461 The DEER 2014-2015 update plan should be released to parties before the end of 2012. As discussed above, Commission Staff should release detailed information on measures, methods, and assumptions that will be the subject of changes within these two future DEER versions as soon as is practical. Staff should not wait until all updates for all measures are completed before releasing information. Instead, Commission Staff should incrementally release information on the details of planned recommended changes as early as practical.
Several parties view the Regional Technical Forum as a model that could provide insights into ways to improve our ex ante updating process. Although we do not change the existing process at this time, we direct Commission Staff to review the processes in other jurisdictions and make recommendations for improvements to our process for consideration prior to beginning the ex ante update for the post-2014 cycle.
Parties commented on the following areas of the non-DEER workpaper review process: retirement of specific non-DEER workpapers, updates of non-DEER workpapers that are covered by the DEER update,462 updates of non-DEER workpapers not covered by the 2011 DEER update - to reflect the 2006-2008 evaluation results,463 the workpaper review process, and mid-cycle measures in the Phase 2 workpaper process.
PG&E comments that where any of the specific parameters of an IOU installation differ from the assumptions that form the basis of the DEER measure, the IOUs will necessarily have to develop a workpaper to convert and/or apply the DEER assumptions to the particular installation. Therefore, PG&E asks the Commission to specify that whether a measure is considered a DEER or non-DEER "measure" not be determined solely based on the technology installed, and that rather the IOUs should prepare non-DEER workpapers for measures where any of the installation parameters differ from the parameters in the DEER update.
The utilities have always had the flexibility to provide a workpaper for a measure that is not in DEER for Commission Staff review, therefore PG&E's request is moot. To minimize the proliferation of workpapers, though, the IOUs are instructed to use DEER values as starting points and/or apply the DEER methodologies for estimating the non-DEER parameter value for cases in which any of the specific parameters of an IOU installation differ from the assumptions that form the basis of a DEER measure. The utilities will not have the option to replace DEER assumptions and values with their preferred values unless the Commission Staff agrees with their proposal for such replacements.
PG&E states that after the DEER is updated, each DEER value or parameter that has been updated should be identified, and a clear procedure to apply these updates to non-DEER workpapers should be established. We agree. We direct Commission Staff and the utilities to work together to identify each of the values that have been updated and develop a clear procedure for applying the updates to relevant non-DEER workpapers. The procedure must follow our intent to utilize DEER assumptions and values in non-DEER workpapers, but we provide Commission Staff flexibility to interpret the details of this requirement in a manner it finds reasonable and practical.
17.1.2.3. Updates of Non-DEER Workpapers not
Covered in the 2011 DEER Update to
Reflect 2006-2008 Evaluation Results
PG&E requests that the Commission clarify that the IOUs should update the High Impact Measures workpapers in accordance with the 2011 DEER update, as opposed to the 2006-2008 evaluation results. In D.09-09-047 and D.11-07-030, the Commission stated that non-DEER workpapers were to be updated with the latest information available. Consequently, the utilities have already been instructed to update with the latest information available, which would include the Commission's 2006-2008 evaluation results. We once again instruct the IOUs to update non-DEER workpapers with the latest information available, including the Commission's 2006-2008 evaluation results, and not wait for future DEER updates before complying with this Commission directive. In the absence of existing DEER values, followed by the utilities shall use the 2006-2008 evaluation results as inputs, when applicable. We leave to Commission Staff to approve the utilities' proposals as to which workpapers require updating. This direction is consistent with the expectation that the best available information will be used when calibrating goals, estimating savings, and reporting results.
The utilities shall submit their non-DEER workpapers as part of their 2013-2014 transition portfolio applications, and each utility shall upload its non-DEER workpapers to its respective directory in the Workpaper Project Archive on the website: http://www.deeresources.info.
Given the limited time available for the utilities to develop and the Commission to approve 2013-2014 transition portfolio applications, PG&E464 and SCE465 state that Commission Staff should first review High Impact Measure non-DEER workpapers filed with the applications and turn to non- High Impact Measure workpaper review as time permits. We agree, and direct the IOUs to provide in their applications a "Non-DEER Workpaper Summary List" that identifies those non-DEER measures they forecast to be High Impact Measures. Commission Staff shall review as many of the workpapers as time allows, beginning with the High Impact Measures, and provide recommended adjustments to the workpapers it has reviewed in a document similar in format to Attachment A of D.11-07-030 that will be included in the decision approving the IOUs' Applications.
If the IOUs do not agree with Commission Staff's adjustment(s) attached to the proposed decision, they may indicate their positions in comments on the proposed decision approving the 2013-2014 transition portfolios as was done in D.11-07-030. The Decision approving the Applications approves any workpapers that are not reviewed, and Commission Staff may review any of these in the future and apply any upward or downward adjustments (consistent with the dispute resolution process described in the following section) on a prospective basis on contracts entered into by the IOU(s) for the relevant measures (i.e., the adjustments will only apply to contracts signed after Commission Staff communicates to the utility that it has selected the workpaper for Phase 2 Mid-Cycle review, as described in D.11-07-030 and amended per the discussion in the following section).
PG&E,466 SCE,467and Ecology Action468 comment on a need for timely Commission Staff Phase 2 workpaper review. PG&E adds that Commission Staff's "conditional approval" review disposition designation has resulted in delays in introducing products into programs.
To address these concerns, to eliminate the potential for multiple iterations of workpaper discussions on disagreements between Commission Staff and the utilities, and to provide an opportunity for disputed values to be vetted in a more transparent manner, we replace the "conditional approval" designation in the Commission Staff review disposition process with the following Phase 2 workpaper approval dispute resolution process that shall commence with the 2013-2014 transition portfolio:
a. If Commission Staff agrees with the parameters included in a non-DEER workpaper for a new measure provided by an IOU, Commission Staff will communicate this to the IOU via email and upload it to the Workpaper Project Area on the http://www.deeresources.info website, and the workpaper will become effective on that date.
b. If Commission Staff disagrees with or needs more information regarding parameters included in a non-DEER workpaper, Commission Staff will recommend revised parameter values (or request additional information) within 25 days of receipt of a work paper with all necessary information provided by the utility.469
c. If the utility finds the revised parameter values unacceptable (and/or any subsequent information exchange does not resolve the disagreements in parameter values), Commission Staff and the IOU will hold one or more meetings to come to an agreement. If agreement on workpaper parameters is reached through this process, Commission Staff will upload the workpaper to the Workpaper Project Area on the http://www.deeresources.info website, and the workpaper will become effective on that date.
d. Every six months, and for each applicable IOU, Commission Staff will develop a draft resolution that identifies the disputed ex ante values proposed by the IOU for each non-DEER workpaper submitted during the previous six months that remains in dispute, along with Commission Staff's recommended adjustments and its rationale for those adjustments. The IOUs may articulate their disagreements with Commission Staff's proposed adjustments in their comments on the draft resolution, and the resolution will be subject to a Commission vote.
We currently require that Commission Staff review all utility proposed non-DEER assumptions and values.470 The utilities must cooperate and collaborate with Commission Staff during the review of the non-DEER workpapers so that the Commission is able to fulfill its oversight responsibilities.471 The process for utility non-DEER workpaper submittal, review and approval shall be as follows:
· Non-DEER measure ex ante values based upon 2010-2012 IOU workpapers shall be updated with the latest available information, including the Commission's 2006-2008 evaluation results.
· Non-DEER workpaper measures that are included in the 2013-2014 DEER update shall be retired in favor of the updated DEER values. Commission Staff with help from the utilities will identify which of the non-DEER workpaper measures are now in DEER and will be retired.
· Non-DEER workpapers that are based on DEER values or methods covered by the 2013-2014 DEER update or that include measures not covered by the 2013-2014 DEER update shall be updated, giving priority to High Impact Measures.
· If a large amount of non-DEER workpapers are received in the 2013-2014 portfolio applications, such that Commission Staff is unable to review them all in time for approval in the 2013-2014 portfolio applications, any workpapers that are not reviewed will receive "interim approval," and Commission Staff may review any of these in the future and apply any adjustments on a prospective basis.
· Commission Staff's review of "interim approval" workpapers or new workpapers submitted mid-cycle shall adhere to the Phase 2 workpaper review process, including the dispute resolution process described herein.
The utilities and other parties point out that there is an inextricable link between the development of DEER and non-DEER measures and that any difficulty in one process (DEER versus non-DEER) is detrimental to the other. SDG&E presents the example that savings for a seemingly simple Compact Fluorescent Lamps measure must actually be calculated using "various combinations of measures vis-à-vis climate zones, home vintages etc. and are required to be analyzed along with the preparation and approval of the corresponding workpapers."472 SCE believes that the level of review to which the DEER is subject is less rigorous than the level of review and scrutiny applied to IOU workpapers by Staff's Data Management and Quality Control (DMQC) consultants.473 SCE notes that about one hundred of its workpapers have not been reviewed and most were submitted more than twelve months ago.474 Portland Energy Conservation Incorporated (PECI) "commends the [Commission] for recognizing that many DEER measures have become out-of-date" but also recommends that retirement of measures must be "coupled with updates to the work papers" and that the current backlog of work papers prevents any expectation that approved work papers will be available when DEER measures retire.475 SDG&E suggests there is a misconception that energy savings values can simply be looked up in DEER. Rather, it contends that many common measures must be further developed in workpapers, and that, "[t]his is not described in the documentation and results in a significant administrative burden and confusion, especially for third party implementers."476
We agree with comments that point out that non-DEER ex ante values will often depend upon DEER. We expect the development of non-DEER values to utilize DEER assumptions, methods and data whenever appropriate. We disagree with the claim by SCE's claim that its non-DEER workpapers undergo a more detailed review and commenting than DEER. Commission Staff review workpapers based upon the expected contribution of the measures contained within those workpapers to the utilities' portfolios. By establishing DEER as Staff's vehicle to recommend ex ante values, while retaining a non-DEER utility workpaper submission process, we provide a two-tier approach allowing for a complete and robust ex ante value dataset. The first tier is DEER which we expect will be focused upon the measures and activities that contribute the most to the utilities portfolios. The second tier is comprised of the non-DEER as well as the custom measure and project values. We expect Commission Staff, when developing DEER content, to devote a higher level of resources to the determination of savings values for each measure. We also expect Commission Staff to undertake research in support of DEER updates when the existing evaluations results, analysis methods and other research literature are found lacking.
We expect that DEER updates will make maximum use of appropriate evaluation data, methods and results. We expect the utilities, when adding new measures to their portfolios, to utilize due diligence when developing the proposed ex ante values. The ex ante parameters should be developed to represent the expected gross and net savings, costs, and lifetime of the measure. For new measures we expect that the development of ex ante values will entail some research to establish reasonable expected values. We also understand that such research may take time and that it may be desirable to allow new additions to be utilized in the portfolio prior to all necessary research being completed.
We encourage the utilities to pilot promising new technologies and utilize the results of research undertaken during the piloting period to improve the ex ante values. Piloting and ex ante value research for new measures is necessary to ensure the utility portfolios can respond to technology changes and innovations in the future while maintaining accurate impact and cost-effectiveness forecasts upon which budgeting decisions can rely. We expect Commission Staff, in their review of utility proposed ex ante values for new measures, to balance the need for accurate ex ante values with the equally important need to continuously augment the portfolios with new technologies that offer promise. The utilities and Commission Staff should collaborate to perform the needed ex ante value research for new measures while those measures are being piloted in the portfolios. We also encourage Commission Staff not to allow "the perfect to be the enemy of the good," in general but especially in determining ex ante values for new technologies that offer considerable promise and (at its discretion) to consider "risk-sharing" approaches when assigning ex ante values to such measures.
Parties raise several issues related the Custom Project Review Process adopted in D.11-07-030. In addition to issues raised by parties, we review the progress Commission Staff and the utilities have made in implementing the review process. We also examine assumptions relating to the gross realization rate to be applied for planning and reporting custom measures during the 2013-2014 period.
The commenting parties state that the custom ex ante review process is hampered by delay and complexity.477 In particular, SCE asserts that the custom project ex ante process suffers from review paralysis and is not clearly defined.478 SCE recommends that it have the option to verify Commission Staff's (consultants') expertise relevant to the project being reviewed prior to performing review/inspection, and that Commission Staff, rather than its consultants, control the process so as to ensure that unintended biases and/or potential conflicts of interests are avoided.479
EnerNOC recommends, and SDGE/SoCal Gas agree, that a more definitive custom project ex ante review process be developed through a stakeholder workshop prior to commencement of the 2013-2014 transition portfolio.480
PG&E notes that custom measures, by definition, have values determined at the time of project application. It asserts that the IOUs' custom applications should utilize the new DEER data on a prospective basis during the transition period.481
Having reviewed parties' comments in this area, we are not inclined to make revisions to the custom project ex ante review process at this time. As with any new process, we expect that initial implementation issues will arise and need to be resolved as kinks in the process are identified and worked out. We note that the utilities are yet to be in full compliance with D.11-07-039, which is an additional reason for us to not make a change at this time.482
As set forth in the Phase IV Scoping Memo, the custom ex ante review process adopted in D.11-07-030 shall continue in the 2013-2014 transition portfolios. The utilities are directed to ensure that custom measure and project calculation tools or methods are consistent with the adopted DEER values and assumptions as applicable. The utilities shall bring all custom measure and project calculation tools used in the 2013-2014 ex ante calculations into compliance with the 2011 DEER Update. Commission Staff shall develop direction for the utilities to follow for individual custom projects, which may span the 2010-2012 and 2013-2014 program cycles (and thus multiple DEER versions) when moving through the various application stages, to satisfy our requirement that their ex ante values utilize the current DEER version.
The 2006-2008 evaluations published by Commission Staff in 2010 indicated areas where net savings values needed improvement. For many custom project activities, the 2006-2008 evaluation results for gross savings were well below the currently adopted gross realization rate adjustment of 90% (adopted in D.11-07-030) for custom project ex ante reporting. Table 1: 2006-2008 Gross Realization Rates for Evaluated Custom Projects summarizes overall gross realization rate values from the 2006-2008 evaluation reports for the utilities' customized measure and project programs. Our concern grows from our observation that the gross realization rate for these types of projects has fallen from a 2002-2003 evaluation result of about 90%,483 to a 2004-2005 evaluation result of about 80%,484 to the most recent 2006-2008 evaluation result in the range of 70%. We recognize that these values were developed for programs with different customer and project mixes and that the responsibility for program evaluation has shifted from the utilities to the Commission Staff. We also recognize that the economic conditions during these time periods were different. However, this declining trend calls for action to ensure that these activities are cost effective and assist the utilities in meeting our policy objectives.
Table 1: 2006-2008 Gross Realization Rates for Evaluated Custom Projects
Claimed Gross Savings |
Evaluated Gross Savings |
GRR | |||||||
GWh |
MW |
MMT* |
GWh |
MW |
MMT* |
kWh |
KW |
Therms | |
PG&E |
911 |
128 |
53 |
503 |
70 |
40 |
55% |
54% |
74% |
SCE |
822 |
118 |
629 |
91 |
76% |
77% |
|||
SoCalGas |
15 |
11 |
73% | ||||||
SDG&E |
180 |
29 |
3 |
142 |
20 |
2 |
79% |
69% |
69% |
Statewide |
1,913 |
275 |
71 |
1,274 |
181 |
52 |
67% |
66% |
74% |
*MMT is Million Therms
As noted above, in comments the utilities and others claim in their comments that changes have already been made to program rules and implementation activities to raise these values. However, we have not been provided quantitative evidence that supports claims.
Our adopted custom measure and project review process was conceived both to help motivate improvements to the ex ante values for those projects and to motivate the utilities to respond to Commission Staff reviews with appropriate program design changes. We expect the utilities to respond to Commission Staff reviews by taking steps to change the program activities to improve both gross and net results. To that end, we direct Commission Staff to conduct Net-to-Gross (net of free ridership) screenings as part of its ex ante project reviews process. We encourage the parties to put forward proposals for changes to custom programs during the portfolio development process, which may include proposals for pilot programs, aimed at improving Net-to-Gross and gross realization rates. We note that the net to gross ratio for custom programs has held steady at approximately 0.5 in evaluations since 1998485 and expect to see changes in approach that could improve that ratio. Studies conducted over the course of these years have offered multiple strategies to improve program influence and should be considered in proposed changes. Additionally, we direct the utilities to make programmatic changes to their custom programs per the recommendations and findings in recent evaluation studies. However, we retain the current default Gross Realization Rate (GRR) value of 0.90 for use in the 2013-2014 transition portfolio.
All ex ante gross savings calculations must establish a baseline against which the installed measure is compared in order to establish savings. Several parties raise issues about the baseline to be used in calculation or setting ex ante gross savings values.
As a general matter, SCE requests a more clearly defined ex ante review process for calculated projects, including clarifications to "vague requirements for project baselines" and "guidance as to what is expected to document project baselines."486 Apparently unaware of our policy regarding early retirement, several parties comment that the use of code baselines hampers progress toward deeper retrofit savings.487
In cases where a code baseline is appropriate to use under our current policy, parties raise non-compliance issues and voice the concern that the code baseline estimates savings for measures with high levels of non-compliance. For example, the Pool Solutions Group claims that minimum code requirements for pool pumps are frequently ignored by both homeowners and pool professionals.488
SDG&E/SoCalGas provide a cautionary note that for the Residential New Construction program the "standard method for calculating savings has always used Title 24 as the base case to determine savings, therefore it is not clear what energy efficiency savings would be achieved if incentives were provided to meet Title 24."489
In cases where the pre-existing equipment is the appropriate baseline, parties suggest alternative methods to establish baseline. For example, PG&E suggests the DEER approach to establishing the incandescent lamp baseline for Compact Fluorescent Lamps that replace those lamps be replaced with an approach based on lumen service levels rather than field observations of use that include customer choices.490
In D.11-07-030, we adopted an approach to establishing a baseline for ex ante gross savings values.491 This approach requires the review of the evidence related to one of the two baseline choices: (1) the pre-existing equipment used in the early retirement case; or (2) new equipment that is feasible to use and is code-compliant or an industry standard practice. Evidence relating to the reasons for the equipment replacement is used to make the baseline choice.
We note that D.11-07-030 may not reflect our clarification that the compelling evidence standard for the determination of baseline equipment must be applied to both possible outcomes.492 Specifically, D.11-07-030 notes that it is necessary to establish, by a preponderance of evidence, that the program has induced the replacement rather than merely caused an increase in efficiency in a replacement that would have occurred without the program.
We direct Staff to update and distribute to the service list of this proceeding Appendix 1 of Attachment B to D.11-07-030, to incorporate clarifications provided here regarding baseline for gross savings estimates, and to indicate that a preponderance of evidence on the motivation for equipment replacement shall be utilized to determine which of the two baseline alternatives is applied for all gross savings estimates.
As with many ex ante value setting activities, there will likely be cases where there is a difference of opinions among experts as to the interpretation of evidence for baseline determination. Commission Staff should use its ex ante review process to establish guidelines on how to evaluate and weigh different types of evidence for the determination of the appropriate baseline alternative.
Once it is established that the program caused the existing equipment to be replaced early, we need to establish the period of accelerated retirement. In our discussion of DEER updates above, we note that DEER contains values for the effective useful life for many technologies and recommend using one-third of the effective useful life as the remaining useful life until further study results are available to establish more accurate values.493 For the case of program induced early retirement, the remaining useful life of the existing equipment should be used as the starting assumption for the period of accelerated retirement.
As is the case when evaluating evidence for program induced early retirement, evidence for the remaining life and the period of accelerated replacement of the existing equipment can also be reviewed. The use of a DEER remaining useful life starting point for the acceleration period may be replaced. However, this should be allowed only if credible evidence is available to support an alternative value and that evidence leads Commission Staff to deem it more credible than of the adopted DEER values. Commission Staff should develop guidelines for the evaluation of remaining useful life evidence for the replacement of the DEER default values for specific projects and technologies. We provide this flexibility to utilize alternative remaining useful life values, based upon project or technology specific evidence, in place of the DEER adopted values primarily for use in Staff's review of the utilities' custom project and measure ex ante values.
The choice of an early retirement baseline implies that a dual baseline analysis shall be performed. 494 In the dual baseline analysis, the existing equipment baseline is utilized for the first or "early-retirement" period, also referred to as the "remaining-useful-life" period. For the second period from the end of the remaining-useful-life period through the end of the effective useful life of the new equipment, the baseline is set using the replace-on-burnout or normal-replacement equipment. The equipment used as the second baseline in early retirement must be equipment that is feasible to use and would be compliant with code requirements or industry standard practice. Regulations, codes, and standards applied to a baseline should be those that are known to be effective at the start of that baseline period, due to regulatory action that has been taken and will be effective at that future date.
The measure or project cost utilized in an early-retirement case is the full cost incurred to install the new high-efficiency measure or project, reduced by the net present value of the full cost that would have been incurred to install the standard efficiency second baseline equipment at the end of the remaining-useful-life period. Thus, the early-retirement cost is higher than the incremental cost used in a replace-on-burnout or normal-replacement case, only by the time value of the dollar amount of the standard equipment full installed cost, using our adopted cost-effectiveness discount rate to calculate that time valuation. As with all measures, our policy expects that incentives offered for early retirement will not exceed the actual early retirement cost.495
We find merit in the concern voiced by NAESCO that the finances of a deep retrofit activity may require convincing a customer to accelerate retirement of older equipment. However, we are equally concerned that the early retirement may push the customer not to do more than minimal code requirements. Early retirements should follow our policy to minimize lost opportunities and cream skimming.496 We expect efforts aimed at replacing less efficient older equipment with newer better than code or industry standard practice equipment to also pursue deepening the retrofits at those sites by combining lower cost faster payback activities with higher cost longer payback measures.
For new equipment choices that are subject to existing regulations, codes or standards, our current policy provides that the baseline equipment be determined by the regulation, code, or standard requirements. However, there may be instances where there is sufficient evidence or documentation that the efficiency or energy use of equipment that meets the requirements of the regulation, code, or standard does not represent the efficiency or energy use of equipment that is typically installed. In those cases it may be appropriate to assign a baseline that equals or exceeds the typical installation in place of the regulation, code, or standard. As noted in parties' comments, there may also be cases where existing regulations, codes and standards are being either ignored or circumvented. Thus, it may be possible for the typical baseline performance to require higher energy use than would be expected (if the regulation, code, or standard was correctly followed or adequately enforced) or lower than would be expected (if the regulation, code, or standard was typically exceeded). We are not prepared to direct any changes to the current practice relative to baseline assignments for these cases. However, we direct Commission Staff, with input from the utilities and other parties, to develop recommendations on: (1) whether it is appropriate to replace the regulation, code, or standard baseline with a typical installation baseline for use in calculating energy savings; (2) under what circumstances and based upon what kind of evidence such a change could be made; (3) if the change to a typical installation baseline is made, how the baseline parameters should be established for use in setting ex ante values; and (4) if this change is made what are the time and budget implications for both Commission Staff and utilities for both ex ante and ex post savings development. In addition, to design their energy efficiency activities in a way to lift the market to levels above the minimum code requirements and standard practice, the utilities should identify and make recommendations for ways to aid or support code enforcement activities through their energy efficiency program activities.
In the cases when there is no regulation, code, or standard that applies, which would normally set the baseline equipment requirements, the baseline must be established using a "standard practice" choice. For purposes of establishing a baseline for energy savings, we interpret the standard practice case as a choice that represents the typical equipment or commonly-used practice, not necessarily predominantly used practice. We understand that the range of common practices may vary depending on many industry- and/or region-specific factors and that, as with other parameters, experts may provide a range of opinions on the interpretation of evidence for standard practice choice. Here again, we expect Commission Staff to use its ex ante review process to establish guidelines on how to determine a standard practice baseline.
Independent of the baseline selection criteria, we would not expect that new equipment proposed for program incentive support would be simply a like-replacement of the existing equipment in efficiency level, as this would imply either a repair or normal replacement that would not quality as an energy efficiency upgrade, unless: (1) the proposed equipment exceeds standard practice or code, and (2) there is clear evidence that without support, the efficiency level would fall to the standard practice or code minimum.
PG&E raises the proposal that the adopted DEER method for establishing an incandescent lamp baseline for Compact Fluorescent Lamps using observed existing and installed incandescent and Compact Fluorescent Lamps be replaced with a theoretical equivalency based on lamp lighting output level as listed on the product packages. However, PG&E provides no evidence to refute that the DEER adopted method accurately reflects the delivered service levels as experienced at the large number of customer sites inventoried or surveyed as part of the 2006-2008 evaluations. Our policy states that measure and baseline equipment choices must provide equivalent service levels.497 By this we mean reliable service that can be delivered over the life of the product under normal operating conditions. In the case of Compact Flourescent Lamps, we give more weight to the evidence provided by the 2006-2008 evaluations from field observations than the claims placed upon a manufacturer's product packaging.
17.2. Next Steps for Post-2014 Process Reforms
In our guidance to the utilities in A.08-07-021, we "found merit in the proposal of some parties for a `rolling' budget cycle" and directed the IOUs to explore this approach with parties and Staff and submit proposals in their applications.498 However, the utilities have yet to develop concrete proposals for the Commission's consideration. In the Phase IV Scoping Memo, the assigned Commissioner again recognized the importance of exploring reforms to improve the energy efficiency regulatory process:
Having start-stop program cycles, many of which contain the same programs cycle after cycle, seems wasteful, and having to review the entire program portfolio with every new cycle imposes heavy burdens on the Commission, parties, and program implementers.499
This scoping memo identified two conceptual models - "rolling" portfolio cycles and "evergreen" programs.500 A plurality of the parties view both concepts favorably, at least at a high-level.501 For example, WEM notes that rolling portfolio cycles would necessitate a rolling schedule for the evaluation process whereas TURN goes so far as to identify specific programs for evergreen status.502
We see benefit in designing a regulatory process that avoids start-stop cycles, if possible. We believe a process that enables the IOUs, with Commission approval, to make longer-term commitments to strategically important measures (or suite or categories of measures) or program delivery mechanisms could be beneficial to the extent it contributes to our long-term market transformation objectives. However, we believe it is premature to authorize specific programs to continue beyond the defined program cycles, until the record has been further developed to address outstanding questions. Critical details have yet to be explored and set forth in proposal that includes the appropriate criteria for granting or revoking evergreen status, the evaluation process necessary to support these models, and the regulatory approval processes to effectuate them. We reiterate our support for investigating these reforms. In order to make progress in this area, we direct Commission Staff to work with the parties to develop viable proposals for possible implementation in the post-2014 period. In addition, we take initial steps in this decision to prepare evaluation data, reporting, and management for the demands these models would place on our evaluation system.
428 TURN opening comments on DEER at 3.
429 NRDC opening comments on DEER at 1.
430 PG&E opening comments on DEER at 16.
431 SCE reply comments on DEER at 8.
432 PGE, Comment on Phase IV Scoping Memo at. 11.
433 NRDC, Comment on Phase IV Scoping Memo at 7.
434 Synergy Companies, Reply Comment on Phase IV Scoping Memo at 5.
435 DRA, Reply Comment on Phase IV Scoping Memo at 9-10.
436 DRA, Reply Comment on Phase IV Scoping Memo at 10.
437 TURN, Reply Comment on Phase IV Scoping Memo at 2.
438 SCE, Comment on Phase IV Scoping Memo at 7; NAESCO, Reply Comment on Phase IV Scoping Memo at 4; NRDC, Comment on Phase IV Scoping Memo at 7; Efficiency Council, Comment on Phase IV Scoping Memo at 10; Ecology Action, Comment on Phase IV Scoping Memo at 14.
439 Efficiency Council, Reply Comments on Phase IV Scoping Memo at 7; NRDC, Comment on Phase IV Scoping Memo at 8.
440 Ecology Action, Comment on Phase IV Scoping Memo at 14.
441 SCE, Comment on Phase IV Scoping Memo, at 7; Ecology Action, Comment on Phase IV Scoping Memo at 2.
442 NRDC, Reply Comment on Phase IV Scoping Memo, at 3; Ecology Action, Comment on Phase IV Scoping Memo at 14.
443 Synergy Companies, Reply Comment on Phase IV Scoping Memo at 5.
444 Efficiency Council, Reply Comment on Phase IV Scoping Memo at 7.
445 TURN, Reply Comment on Phase IV Scoping Memo at 1.
446 NAESCO, Reply Comment on Phase IV Scoping Memo at 4; SCE, Comment on Phase IV Scoping Memo at 14.
447 TURN opening comments on DEER at 2.
448 PG&E reply comments on DEER at 8.
449 SCE reply comments on DEER at 11.
450 NRDC opening comments on DEER at 2.
451 PG&E opening comments on DEER at 16; SCE reply comments on DEER at 10, respectively.
452 PG&E opening comments on DEER at 16.
453 PG&E opening comments on DEER at 20.
454 D.05-01-055, Section 5.3.2 at 120.
455 Ibid. at 121.
456 Ibid. at 122.
457 D.05-01-055, Section 5.3.2 at 128.
458 Ibid. at 130.
459 These changes are known at least one year ahead of their effective date.
460 http://www.energy.ca.gov/title24/2013standards/prerulemaking/.
461 Commission Staff should target two versions of DEER for our adoption late in 2013: first, the DEER update for use in 2014 reporting that incorporates changes to codes and standards effective by that time, and second, the DEER update to be used for 2015 and beyond planning.
462 PG&E comment on Phase IV Scoping Memo at 11-12.
463 Id. at 12-13.
464 Phase IV Scoping Memo at 14-15.
465 SCE reply comment on Programmatic Guidance Ruling at 6.
466 PG&E Comment on Phase IV Scoping Memo at 13-14.
467 SCE opening comments on DEER at 16.
468 Ecology Action reply comment on Phase IV Scoping Ruling at 4-5.
469 As set forth in the November 18, 2009, ALJ ruling in A.08-07-021, et al.
470 ALJ Ruling in A.08-07-021, (November 18, 2009).
471 D.09-09-047 Order Paragraph 4 states that, "Review of completed IOU workpapers regarding ex-ante savings estimates are subject to Commission Staff review and approval, as set forth in an ALJ Ruling of November 18, 2009 in Application 08-07-021, et al. Each IOU shall cooperate with Commission Staff to allow upfront consultation regarding such workpapers." Discussion in Section 5.2 (page 19) of that decision states that, "We will require the IOUs to cooperate and collaborate with ED in the development of these workpapers."
472 SDG&E opening comments on DEER at 3.
473 SCE opening comments on DEER at 12, 13.
474 Id. at 16.
475 PECI opening comments on DEER at 2.
476 SDG&E opening comments on DEER at 9.
477 PG&E, reply comment on Programmatic Guidance Ruling at 5; NRDC, reply comment on Programmatic Guidance Ruling at 6; EnerNOC, comment on Programmatic Guidance Ruling at 9; SD&GE/SoCalGas, reply comment on Programmatic Guidance Ruling at 4-5; NAESO, Comment on Phase IV Scoping Memo at 5-6); Trane, reply comment on Programmatic Guidance Ruling at 4; Gary Gockel, comment on Programmatic Guidance Ruling at 4.
478 SCE Comment on Programmatic Guidance Ruling at 7.
479 Id..
480 EnerNOC, comment on Programmatic Guidance Ruling at 10; SDG&E/SoCalGas reply comment on Programmatic Guidance Ruling at 5.
481 PG&E, Comment on Phase IV Scoping Memo at 13.
482 See Ordering Paragraph 7 and Attachment B to D.11-07-039. The fact that it was only in February of 2012 that SCE started to provide the required custom projects summary list (for Commission Staff to select projects to review), makes SCE's strong critiques of the custom project ex ante process particularly puzzling.
483 2003 Statewide Nonresidential Standard Performance Contract (SPC) Program Measurement and Evaluation Study, for SCE, December 2005, at ES-1, reports a GRR for source BTU of 0.89 with a 90% confidence interval of 0.83 to 0.96.
484 2004-2005 Statewide Nonresidential Standard Performance Contract Program Measurement and Evaluation Study, Volume 1, for SCE, September 2008, at ES-2, reports a GRR for source BTU of 0.79 with a 90% confidence interval of 0.69 to 0.89.
485 See Section 5.3, 06-08 Final Evaluation Report for PG&E Fab, Process and Manufacturing Contract Group (Itron, February 2010), available at http://www.calmac.org (Study ID CPU0017.01).
486 SCE opening comments on ALJ Ruling regarding program guidance at 7-8.
487 See for example, City of Oakland, opening comments on Phase IV Scoping Ruling at 6, and NEESCO opening comments on ALJ Ruling regarding program guidance at 9.
488 Pool Solutions Group opening comments on ALJ Ruling regarding program guidance at 6.
489 SDG&E/SoCalGas opening comments on ALJ Ruling regarding program guidance at 6.
490 PG&E opening comments on DEER at 17.
491 D.11-07-030, Appendix I to Attachment B.
492 D.11-07-030 at 40.
493 Summary of EUL-RUL Analysis for the April 2008 Update to DEER, KEMA, at 2.
494 EEPMv4, Rule IV.2. and also footnote 9.
495 EEPMv4, Rule IV.4
496 EEPMv4, Rule II.4.
497 EEPMv4, Appendices, at 5, defines energy efficiency as "Activities or programs that stimulate customers to reduce customer energy use by making investments in more efficient equipment or controls that reduce energy use while maintaining a comparable level of service as perceived by the customer."
498 D.07-10-032 at 95-96.
499 Phase IV Scoping Memo at 3.
500 For purposes of this decision, "rolling" portfolio cycles refer to any set of reforms which obviate the need for arbitrary cycles of preparation, regulatory review, authorization, evaluation, and termination of the program portfolio in its entirety. "Evergreen" programs refer to a regulatory scheme in which programs would be authorized to continue, within specified certain parameters and under continuous evaluation and oversight, as long as they meet certain specified criteria.
501 SCE, NRDC, TURN, DRA, Efficiency Council, Commercial Energy California, EnerNoc, OPOWER, CCSF, and WEM support consideration of rolling portfolio cycles. SCE, TURN, Efficiency Council, Commercial Energy California, EnerNoc, OPOWER, LGSEC, and CCSF support the idea of, at least, some evergreen programs. CFC is concerned that these approaches could cause misalignment between the approval of funding and specific programs causing inefficient programs to continue without proper evaluation of success (CFC Comments on Phase IV Scoping Memo at 4).
502 TURN Comments on Phase IV Scoping Memo at 4-5.
