5. Discussion

We are pleased to see the broad range of active participants in this phase of the proceeding, as evidenced by workshop participation, the diverse composition of the Coalitions and Collaborating Parties, and the large number of opening and reply comments. Although these participants clearly do not agree on how to structure the specific Program Choice and Portfolio Management functions, we note that the administrative proposals are more similar than different.

In particular, all of the proposals recognize that the Commission is responsible for the policy oversight and quality assurance functions, and provide thoughtful recommendations on how the Commission might best obtain the policy and technical expertise to assist it in carrying out those responsibilities. All of the proposals recommend approaches to EM&V designed to mitigate potential conflicts between the overall administrator, program implementers, and EM&V contractors. They all propose the use of an EM&V technical advisory group to assist the Commission in this effort. And all of the proposals envision an energy efficiency delivery system in California that continues to include a role for both IOU and non-IOU program implementers, although they differ with respect to how those IOU and non-IOU implementers should be selected. Finally, with the exception of the WEM/SESCO Coalition proposal, all proposals establish one or more advisory groups to provide guidance in program selection and portfolio management, irrespective of what entity or entities perform those functions.

While there are significant areas of agreement, there are also key differences among the parties with respect to the threshold issue of what entity (or entities) should be responsible for Program Choice and Portfolio Management in the future. Based on the proposals and comments in this proceeding, we believe that this major "fork in the road" must be addressed before we can proceed further to design an administrative structure for energy efficiency programs.

Before turning to this issue, we wish to comment on a proposal that was not put forth in the April 8 filings, namely, to continue with an administrative structure that places Commission staff in the role of Program Choice and Portfolio Management. As described above, this structure was put in place as a rapid response approach during the energy crisis, when the Commission perceived a need to play a more significant role in the Program Choice and Portfolio Management functions. We commend staff for performing an admirable job under very difficult circumstances and constraints over the past few years. However, we believe that this current structure should not be continued beyond 2005 for several reasons.

First and foremost, placing our staff in the role of Program Choice and Portfolio Management stretches limited staff resources between those functions and the quality assurance and EM&V responsibilities that we believe should be the primary focus of our staff efforts. While requesting increases in staff resources is always an option, the outcome of those efforts is highly uncertain, particularly given the budget realities in California today. Moreover, even if staff resources were not limited, we are concerned that many innovative programs may not be discovered through an application and review process at a regulatory agency. Past experience indicates that program administrators and program implementers, working together with public input, are well suited to the task of developing innovative and cost-effective energy efficiency programs from concept to full program design for our consideration--examples of which include Flex Your Power, Standard Performance Contracting and Energy by Design.

The City of Oakland and others note that placing the responsibility on this Commission to make the initial selections of energy efficiency programs and then to oversee the portfolio management of those selections puts us in the position of both "judge and jury."⁵³ We believe that the Commission could fairly perform the program selection function, while at the same time overseeing quality assurance and an EM&V process that provides us with an independent assessment of program performance, but on balance we agree that separating these two functions promotes more confidence in the process and is a better use of staff expertise.

Finally, we concur with Commissioner Kennedy's observations that:

"The Public Utilities Commission is a regulatory agency, not an administrative agency. As such, the Commission's regulatory functions, and the Commission's responsibility for providing independent oversight of all ratepayer-funded programs, are incompatible with administration of any of those programs or contracts on a long-term basis."⁵⁴

We now turn to the threshold issue on administrative structure in this proceeding.

5.1. Threshold Issue: Who Should Perform the Program Choice and Portfolio Management Functions?

We concur with the observations of our Strategic Planning Division and the Regulatory Assistance Project that there is no single best model for how energy efficiency programs should be administered, particularly with respect to the Program Choice and Portfolio Management functions. One size does not "fit al": The best administrative structure depends on each state's particular context.⁵⁵ We believe that these questions should be carefully considered within the specific context of California, beginning with the goals for energy efficiency that we have established in the Energy Action Plan.

As discussed in Section 3, energy efficiency has been a component of energy planning in California since the mid-1970s, but the focus and goals have shifted over time. Prior to the Collaborative era, energy efficiency programs were developed outside the IOUs' resource planning process. As a result of the Collaborative and subsequent DSM rulemaking, energy efficiency was recognized as an integral component of utility resource procurement and an important means of achieving the Commission's goal of "reliable, least-cost, environmentally sensitive electricity service."⁵⁶

This focus shifted dramatically in 1996 to reflect the competitive framework that this Commission and Legislature envisioned for the electric
services industry at that time. Ratepayer investments in energy efficiency were undertaken to develop a fully competitive market in energy efficiency services so that customers could seek and obtain these services in the private sector. The goal was to provide ratepayer funding for this purpose for a transitional period only (through December 2001), at which time all ratepayer funding of non low-income energy efficiency programs would cease.

Today, the Energy Action Plan has placed energy efficiency back at the forefront of resource procurement activities in California. In particular, the plan establishes a loading order of energy resources that requires California to first optimize "all strategies for increasing conservation and energy efficiency to minimize increases in electricity and natural gas demand" before turning to supply-side resources.⁵⁷ With the return of the IOUs to resource procurement and the policies articulated in the Energy Action Plan, the focus of energy efficiency in California has returned to resource acquisition. Consistent with that shift in focus, in D.02-10-062 we directed the IOUs to optimize electric energy efficiency investments in their resource plan portfolios for our consideration, regardless of the limitations of funding through the PGC mechanism. Based on our consideration of those projections, we increased energy efficiency funding to

over $800 million for the 2004-2005 funding cycle, or an average of approximately $400 million per year. In addition, we recently augmented natural gas energy efficiency funding for PG&E, SDG&E and SoCalGas on an
expedited basis, in order to expand current program offerings for the 2004/2005
winter season.⁵⁸ By D.04-09-060, we established aggressive natural gas and electric savings goals by IOU service territory through the year 2013, subject to updates for 2009 and beyond. As described in that order:

"For the three electric IOUs, today's adopted savings goals reflect the expectation that energy efficiency efforts in their combined service territories should be able to capture on the order of 70% of the economic potential and 90% of the maximum achievable potential for electric energy savings over the 10-year period, based on the most up to date study of that potential. These efforts are projected to meet 55% to 59% of the IOUs' incremental electric energy needs between 2004 and 2013.

For natural gas, our adopted savings goals are designed at this time to capture approximately 40% of the maximum achievable potential identified in the most recent studies of that potential. This level of expectation recognizes the fact that natural gas program funding levels have dropped substantially over the last five years, and that ramping up those efforts to meet the full savings potential may take more time than on the electric side. It also recognizes some uncertainty over the level of achievable savings in the non-core sector. Nonetheless, today's adopted natural gas savings goals represent substantial "stretch goals" by anyone's standards: They reflect an increase in savings by 244 Mth over the 210 Mth in savings that would be achieved if current funding levels and program effectiveness (therms per dollar) remained constant. In other words, today's adopted goals for natural gas energy efficiency represent a 116% increase in expected savings over the next decade, relative to the status quo."⁵⁹

No state has ever placed energy efficiency at the forefront of energy policy in this manner, or has committed the level of resources that California has to meet the goal of optimizing energy efficiency investments. It is therefore imperative that we adopt an administrative structure that is capable of mobilizing the resources and efforts needed to meet the goals of the Energy Action Plan, without uncertainty or delays that could undermine California's ability to recover from energy crisis and move its economy ahead with "reasonably priced and environmentally sensitive energy resources."⁶⁰

To this end, our options for energy efficiency administration should be considered in the context of California's regulatory framework for resource procurement as it exists today and for the foreseeable future. As described in previous Commission decisions,⁶¹ the energy crisis of 2000 and 2001 changed the regulatory landscape in a profound way for California. The Commission and the Legislature responded to the crisis in 2002 with direction to the IOUs that they were to resume responsibility for procuring resources to meet customer demands, subject to Commission oversight and approval.⁶² In contrast to Texas and other states that have implemented full retail competition, California IOUs are required once again to plan for and acquire both supply-side and demand-side resources for a large portion of their natural gas customers and all of their electric customers. Even under the core/non-core structure for electric customers currently under consideration, the IOUs would remain responsible for resource procurement for a sizeable level of electric load.

This is very different from the approach taken in Texas, for instance, where the IOUs are not allowed to participate in planning or delivering energy services (supply- or demand-side) within their service territories, except to oversee standard contracts with third-party providers. In Texas and other states that have implemented full retail competition, decisions concerning the optimal levels of energy efficiency and supply-side resources are determined entirely by the private market.

In California, these decisions will now be made in the resource planning process undertaken by the IOUs, subject to our oversight and approval. In this context, establishing an independent administrator (or administrators) for program choice and portfolio management means that all of the program selection and day-to-day management decisions are "handed down" to the IOUs to incorporate into their resource plans and resource adequacy projections (TURN/ORA Coalition), or left to the private market to determine (WEM/SESCO Coalition). While this may not be an issue in other states where the IOUs are not as involved--or not involved at all--in resource procurement, we believe it is an unworkable approach to integrated resource planning in California. In particular, as we stated in D.04-01-050, California IOUs should not be required to adopt the forecasts and resource plans of others -because "[w]e strongly believe that the utilities themselves must be responsible and accountable for providing their customers reliable service and just and reasonable rates; this is the utilities' statutory obligation to serve."⁶³ And, as discussed below, our experiences in California have left us unwilling to rely solely on competitive market solutions to meet customers' energy needs.

Even if the IOUs were not responsible for resource procurement, we would have significant concerns about the degree of control we could exert over third parties under an independent administrative structure. The Commission has broad regulatory authority to ensure and enforce the IOUs' compliance with our policy rules and requirements based on current statute and Constitutional authority.⁶⁴ In contrast, the proposals for independent administrators in this proceeding rely on contractual authority. This form of authority is potentially weaker, more complex, and less flexible than relying on our regulatory powers. In particular, we would have limited recourse in the event that the programs do not deliver the requisite energy savings or the program administrator fails to perform in other ways. As NRDC points out, the remedies for breach of contract are much more limited than our regulatory authority under current law:

"If a contracting party fails or refuses to discharge his/her contractual obligation, a `breach' of contract occurs. However, the standard as to whether there was in fact a breach in contractual performance is much higher than a CPUC regulatory determination. In order to recover for damages for a breach of contract, a breach has to be `material.' Material breach occurs if the defect in the promisor's performance seriously disappoints the reasonable expectation of the aggrieved promisee. The burden of proving that this `impact is serious' is on the promisee who claims the privileges accorded to the victim of material breach. And when it is found that a party breached a contract, the Government agency is limited to usual remedies for breach. [Footnote omitted.] These remedies include restitution and monetary damages. Remedies for a breach of contract do not include punitive or exemplary damages. In addition, in most instances, specific performances are not available. On the other hand, the CPUC's regulatory authority allows it to order specific performance, and/or impose fines and penalties if the utilities do not perform."⁶⁵

TURN and CCSF argue that "there is ample opportunity and authority for the Commission to exercise any degree of control it desires over a non-utility administrator through a contract.⁶⁶ While the Commission can and does exercise control of contractors by defining the terms and scope of work, entities that sign contracts with the Commission do not by that fact alone become subject to the Commission's regulatory jurisdiction. In fact, according to the presumed contract terms under the TURN/ORA Coalition proposal, the only expedient remedy for unsatisfactory performance is the termination of the program administrator's contract.⁶⁷ Any other remedy could require us to litigate the matter in Superior Court, which is time consuming, expensive and uncertain, and less satisfactory than direct regulation.⁶⁸ In either case, we would be forced to assign an interim administrator, a scenario that could be highly disruptive and costly. In order to meet our goals for energy efficiency, we must have the authority to hold the administrator(s) fully accountable for delivering energy savings without recourse to litigation. We believe that this authority is clearly established with our regulatory oversight of the IOUs, but considerably less certain under the proposals for independent administration in this proceeding.

Returning the IOUs to the program choice and portfolio management roles for energy efficiency is also a logical corollary to the market structure we have recently adopted for supply-side resource procurement. It is instructive that the debate over this issue in our Procurement Rulemaking focused in large part on the same threshold issue in this proceeding: The role of IOUs as both the "program choice/portfolio manager" and a potential "implementer" of supply-side resources, e.g., through dispatch of existing IOU resources or IOU construction of new power plants. Parties to our Procurement Rulemaking lined up on different sides of this issue, as they have in this proceeding.

For example, TURN, the Independent Energy Producers Association (IEP) and the Western Power Trading Forum (WPTF) expressed many of the same concerns about the potential for conflicts of interest discussed by TURN, ORA, WEM, SESCO and others in this proceeding if the IOU selects and manages the portfolio of supply-side resources and also "contracts with itself" for power production from existing or new generation. To address this issue, TURN, IEP, WPTF recommended two alternatives. First, that there be "independent administration" of the bid preparation and selection process if IOUs are allowed to participate in the solicitation with IOU-owned and/or IOU-constructed new plant. In the alternative, these parties recommend that the IOUs be required to administer an open competitive solicitation with third-party market generators but not be allowed to compete in the solicitation with IOU proposed new plant.⁶⁹ Third-party developers supported this position by arguing (as proponents for independent administration do in this proceeding) that separating IOUs from resource selection process coupled with the discipline of an open competitive solicitation is the best way to ensure lower costs and risks to ratepayers.⁷⁰

In contrast, SDG&E and the IOUs recommended a supply-side market structure that would allow the IOUs to directly participate in resource selection as well as in the ownership of new generation facilities. In doing so, they presented many of the same arguments that the IOUs, NRDC, LIF and others present in support of their preferred energy efficiency administrative structure in this proceeding. These include: (1) the stability and permanence of a regulated utility; (2) the ability of the Commission to directly regulate the price, terms and quality of the generation service provided by the utility; and (3) the availability of a proven high-quality workforce (both management and labor).⁷¹

In weighing the arguments on market structure for long-term supply side procurement, the Commission concluded that California "should not rely solely on competitive market theory and the behavior of market generators", noting that our State has "a long history of reliable service being provided by utility-owned and operated generation plant and a recent painful history of rolling blackouts and high price spikes from reliance on third-party generators in a


poorly designed competitive market."⁷² In view of these overriding concerns, and in recognition of certain benefits of allowing IOUs to participate in owning new generation facilities, the Commission rejected proposals to either (1) remove IOUs from the role of resource selection or (2) allow the IOUs to select supply-side resources but not to participate as implementers. Instead, the Commission determined that IOUs should participate in both functions "in order to have the assurance of more state control over resources and an effective check against competitive market manipulations and abuses."⁷³

Accordingly, in D.04-01-050, the Commission established a market structure that placed the California IOUs in the role of program choice and portfolio manager of supply-side resources (including dispatch decisions for IOU-owned generation plant), but also allowed them to directly participate as supply-side implementers by owning and/or building new generation facilities. In setting the market structure and rules for long-term procurement, the Commission recognized that it would need to be vigilant in overseeing "that no perceived bias occurs in selecting, or dispatching the resources, especially when the current cost recovery mechanisms favor the rate-based power plants." To this end, the Commission put in place important safeguards to "provide assurance to the third-party generators that we see a meaningful role for them in California's energy future."⁷⁴ The Commission found that third-party generating capacity, "if contracted properly,...holds a number of advantages for California ratepayers."⁷⁵

More specifically, to address concerns that IOUs would rather rely on their own existing resources than on those that come from the market, the utilities are monitored for their patterns of dispatch to assure that the operations are undertaken in a least-cost manner.⁷⁶

In addition, the Commission directed the IOUs to solicit future long-term generating capacity resources from non-IOU suppliers through a formal RFP process as a "standard procedure."⁷⁷ The Commission established Procurement Review Groups (PRGs) comprised of eligible non-market participants to consult with the IOUs in the design of the RFP and the evaluation of bids on a quarterly basis.⁷⁸ At the same time, however, the Commission recognized that IOU-owned and/or built projects should not be discouraged "where they are cost-effective and appropriate."⁷⁹ Accordingly, IOUs were permitted to present such projects for the Commission's consideration outside of the RFP process with evidence and justification for why IOU ownership structure is preferable, and how cost containment would be addressed.⁸⁰

The language of D.04-01-050 is instructive on how the Commission viewed the need for safeguards in the IOUs relationship with third-party implementers on the supply-side, and the best way to put such safeguards in place:

"WPTF argues for a specific structure for capacity procurement that puts procurement via contract on an equal footing with utility-built options. WPTF's proposal is that prior to its issuance, an RFP must be approved by the Commission or an independent third party to verify that it is not tilted in favor of the utility or its affiliate's bid. Second, bids should be evaluated by an independent third party, such as an accounting firm, consultant, or specially convened review panel. Finally, the third party will select a winning bid which, if it meets the criteria presented in the RFP, the utility must accept.

"WPTF's proposal would result in a cumbersome process, and one that would be difficult for any utility to endorse, especially as it reserves final choice of contracting partner to a party other than the utility itself. But its need derives from the perception that without the involvement of independent parties in the development of the RFP, the evaluation of the bids, and the ultimate selection of the winning bidder, the utility would have an incentive to act in ways that would bias the process in favor of itself.

"The Commission currently has in place safeguards to address WPTF's concerns. First, each utility has a Procurement Review Group (PRG) that consults with the utility in the design of the RFP and the evaluation of bids. Next, the Commission will review all long-term commitments that result from an RFP through its formal process which allows notice to all parties and an opportunity for public review and comment. Based on our continuing review of the RFP process, we will adopt additional safeguards if we find it is necessary."⁸¹

As an additional means of addressing potential bias in this structure, the Commission endorsed the concept of creating a procurement incentive mechanism:

"The utilities have an opportunity to invest and earn a return from generation assets; a similar opportunity for profit should be provided for selecting and managing well all other procurement products."⁸²

"The goal of this effort is to motivate the IOUs to procure least-cost supply-side resources and make cost-effective demand-side investments, taking into account the environmental costs (or benefits) of various resource options. Our challenge will be to create an overall procurement incentive framework that aligns the interests of utility investors, management and ratepayers such that the proper balancing of these preferred resources occurs in the procurement of power from existing and new resources."⁸³

In sum, placing IOUs in the role of program choice and portfolio management, as proposed by the IOUs' Coalition and the NRDC/LIF Coalition, is consistent with the "hybrid" market structure we established in the Procurement Rulemaking for supply-side resource acquisition. This structure consists of both IOU and non-IOU market participants in the ownership and construction of supply-side resources. Project selection is the responsibility of the IOUs as part of their overall resource procurement obligations. The process is subject to Commission oversight and the safeguards described above to ensure against IOU bias in the selection process.

In contrast, the independent administrative structure proposed by TURN/ORA Coalition would create a new organization (and in the case of WEM/SESCO, several competing organizations) to perform the corollary functions of program choice and management for energy efficiency, while leaving the IOUs responsible for those same functions for all supply-side resources. We do not see any clear advantage to creating this dichotomy in the context of California's current resource procurement structure. As discussed above, we rejected the principle that no entity should be allowed to assume both the program choice/portfolio management function and implementation function in our Procurement Rulemaking for supply-side resources. While TURN, ORA, SESCO, Ecology Action, WEM and other proponents of independent administration clearly consider this principle to be paramount, we do not view it to be an end unto itself. Returning the IOUs to these administrative functions has other advantages, as discussed above. Moreover, we believe there are significant impediments to putting independent administration in place that will introduce considerable delay and uncertainty into the process, thereby undermining California's ability to achieve the Energy Action Plan goals.

Our unsuccessful attempts to shift to independent administration during electric industry restructuring created over two years of uncertainty in the administration of energy efficiency programs, an experience that we cannot afford to repeat at this critical juncture for energy procurement in California. That experience persuades us that we should carefully consider the potential legal and implementation challenges inherent in moving to the independent administrative structure proposed by the TURN/ORA Coalition or the WEM/SESCO Coalition, even without a "CBEE" type board overseeing the RFP solicitations. In their April 8, 2004 filings, both the TURN/ORA Coalition and the WEM/SESCO Coalition appear to contemplate the transfer of ratepayer funds from the IOU to the independent administrator(s). For example, the TURN/ORA Coalition states that since the Legislature directed the Commission to oversee the expenditure of PGC funds collected for energy efficiency purposes, the Commission may "order the utilities to collect the PGC energy efficiency funds and to transfer them to an independent entity."⁸⁴ The WEM/SESCO Coalition echoes this argument and concludes that no legislation would be needed to implement their proposed administrative structure. ⁸⁵

This was also our position in 1998 regarding our authority to oversee PGC funds collected pursuant to Section 381, as well as the telecommunication public purpose funds, which were collected from the IOUs, kept in trust accounts and spent under the Commission's ultimate authority. As described in Section 3.2, the Department of Finance and the Attorney General rejected this position. Both the Attorney General's and the Department of Finance's representatives stated that the ratepayer money such as the PGC is public money that can be held by the IOUs and spent under Commission direction, but in the absence of specific legislation, cannot be moved to an outside trust account or bank account.⁸⁶ Thus, if funds are not held by the IOUs, they must be held in a treasury account, or other account specifically authorized by the Legislature.⁸⁷

The WEM/SESCO Coalition proposes that the System Director (which would be the CPUC, or its consultant, or a non-profit) accept applications from potential administrators, which would then contract with program implementers using a standard offer system.⁸⁸ The proposal states that the Commission "might allow IOUs to temporarily hold the funds, then pay them to the System Director and/or Administrators."⁸⁹ The WEM/SESCO Coalition further argues that "The Legislature has made the CEC the fiscal agent for some renewable resource funds collected by the IOUs, and we see no barrier to a CPUC order directing the IOUs to pay the funds, as collected, to the CEC or other governmental entity..."⁹⁰ The barrier is that the Legislature must authorize such a transfer, as it did for PGC funds for renewable resources.⁹¹ During oral argument and in the subsequent briefs, the TURN/ORA Coalition and WEM/SESCO Coalition proposes that the IOUs pay the program administrator(s)' bills at the direction of Energy Division, rather than actually transferring IOU funds to program administrators. As these parties note, we have authorized Energy Division to enter into contracts for EM&V and other specialized energy efficiency-related activities (e.g., audits) for which the IOUs pay the contractors directly.⁹² We anticipate that Energy Division will enter into future contracts for EM&V that will be similarly managed and funded, consistent with today's adopted EM&V administrative structure. (See Section 5.3.) This payment arrangement does not appear to raise the same concerns that troubled the Attorney General and Department of Finance in 1998, since ratepayer funds are not moved to an outside trust fund or bank account, but are held by the utilities, and then expended for approved expenses.

In considering this approach to funding a non-IOU program administrator or administrators, it is important to understand that such contractual arrangements require Energy Division to review and approve each invoice submitted by the contractor. After that review, if the invoices are found to be reasonable based on the scope of work and work product, Energy Division sends a letter informing each IOU of its share of the contractor's bill, which the IOU then pays directly to the contractor. While this arrangement has functioned well


for the specialized contracts authorized to date, we do not believe it is feasible or desirable to extend this approach to program administrators, given the magnitude and broad range of tasks associated with the Program Choice and Portfolio Management functions.

Effective staff contract management requires that the contract manager review monthly invoices to ensure that expenses are reasonable and within the approved scope of work, prior to authorizing payments. Requiring our Energy Division staff to extend this type of contract management to a program administrator (or multiple administrators) responsible for an annual budget of over $400 million raises the same issues of resource allocation we discussed earlier. While we will need to closely oversee the administration of energy efficiency programs under any administrative structure, returning the Program Choice and Portfolio Management functions to the IOUs does not require Energy Division to approve monthly invoices in order to maintain effective oversight of program expenditures. Instead, this Commission (with support of its staff) can review reports on program accomplishments and other EM&V documents, and make adjustments for unreasonable program expenditures in our regulatory proceedings, as appropriate.

In sum, even if we desired to pursue a model that transfers funds from the IOUs to an outside entity, we would first need to seek legislation similar to the provisions that authorize the transfer of telecommunication public purpose funds to treasury accounts, or PGC funds to the CEC treasury accounts. This would delay our ability to move forward with a permanent administrative structure for energy efficiency until the Legislature (and Governor) enacted a
statute. Moreover, the outcome of that process would be highly uncertain. We note that once those funds are in the state treasury, then state contracting rules apply.⁹³ Once in the state treasury, the funds would also be more easily available for "borrowing" for other purposes. Our own experience in California, and the experience of other states, indicates that the resulting funding uncertainty could be significant. ⁹⁴ For the reasons discussed above, requiring the IOUs to pay the bills of non-IOU program administrator(s) subject to Energy Division review of invoices does not, in our opinion, represent a viable alternative to transferring IOU funds to an outside administrator, absent statutory authority.

The administrative structure proposals presented by the TURN/ORA Coalition and WEM/SESCO Coalition, both of which contemplate the Commission entering into contracts with outside entities to perform energy efficiency functions, also face the risk of legal challenge as being inconsistent with Government Code § 19130(b). This section requires that civil servants
perform the work of the State. To date, the contracts for specialized energy efficiency activities, such as EM&V, have generally involved very specialized
expertise and have not been challenged. However, the magnitude, broad scope and nature of administrative tasks performed by the program administrators under TURN/ORA Coalition proposal-requiring 25 professional employees, which we believe is a conservative estimate-will make it a more likely target for legal challenge.

TURN and CCSF argue that this is a legal challenge we could win. ⁹⁵ They note that, while the SPB Executive Director ruled against the Commission with respect to independent administration during the late 1990s, the matter was settled before the full SPB ruled on the matter. In their view, the Commission could prevail on this issue before the full SPB by successfully arguing that the work of the independent administrator(s) falls within at least one exemption from the Government Code § 19130(b) requirement. However, even if we were to prevail on a challenge to our contract with the program administrator(s), the delay and uncertainty associated with any challenge would interfere with our ability to aggressively deploy energy efficiency programs and meet our energy efficiency savings goals for 2006 and beyond. Moreover, despite TURN and CCSF's confidence, we cannot be certain of a favorable SPB ruling.

In addition to the legal risks described above, the administrative proposals recommended by TURN/ORA Coalition and WEM/SESCO Coalition create other substantial implementation challenges. Developing and overseeing the RFP for program administrator (or multiple RFPs under the WEM/SESCO Coalition proposal) is a significant and time-consuming task. During restructuring, our effort to develop an RFP for an administrator of market transformation programs (which would not be transferable to today's resource procurement environment and energy efficiency goals) took many months to accomplish and involved Commission staff, CBEE board members and a team of technical and administrative contractors. Moreover, that effort of start-up involved a program of considerably smaller size and scale-approximately half the level of annual funding that we currently authorize. We would expect at least that amount of time would be needed to develop an RFP and solicit bids for administrator(s) under the TURN/ORA Coalition and WEM/SESCO Coalition proposals, and another three to six months for Commissions staff to evaluate the proposals, select the winner(s) and obtain Commission approval of the selection.

Start-up of this structure would also be an expensive undertaking. The TURN/ORA Coalition proposal estimates that independent administration of the Program Choice and Portfolio Management functions would require 25 full-time professional staff. We concur with the assessment of ICF Associates that experience in other states indicates that this estimate is extremely conservative. The Energy Trust of Oregon, for example, operating within a structure very similar to that proposed by TURN/ORA Coalition, currently employs approximately 28 staff to administer an energy efficiency program 1/10^th the size of California's programs.⁹⁶ Although there certainly will be economies of scale, it is difficult to comprehend how this structure could be developed and managed with 25 staff, particularly within the substantially larger and more diverse program environment in California. This problem is magnified when multiple entities are performing the administrative functions, as proposed by the WEM/SESCO Coalition. Not only would each administrator require staffing to oversee the standard offer contractors and perform general administrative duties and responsibilities required by the Commission (e.g., tracking expenditures and performance, reporting requirements), but the Commission would need to increase its staff to oversee contracts with potentially dozens of individual administrators throughout the state.

Moreover, each of these proposals fails to recognize the huge fiduciary responsibility that would need to be assumed by the program administrator or administrators. These entities would be responsible for management of over $400 million annually. The required level of fiscal control and the business systems needed to support that control are complex and expensive. As ICF Associates points out, there are few, if any existing non-IOU organizations with an understanding of the energy efficiency business that have managed anywhere near that level of funding. Even if organizations responding to the RFP for program administrators demonstrated the required level of fiscal control and business systems, we are skeptical that they could accomplish these functions with a total staffing of 25.

This brings us to a critical question concerning independent administration: Are entities of sufficient capability and experience to administer energy efficiency likely to respond to the RFP? The ORA/TURN Coalition proposal places a great deal of emphasis on having the administrator be a "single-purpose" entity, in order to ensure that energy efficiency is the core focus of the administrator. We interpret this to mean that the single organization or partnership of firms forming the administrator would be disqualified if they were engaged in any businesses that create a conflicting financial interest. However, as NRDC points out, many of the entities in California that are most qualified to form the administrator under the ORA/TURN Coalition model engage in multiple activities in the energy industry. Thus, while a "single purpose" entity sounds simple and appealing in theory, we expect that it could be very difficult to implement in practice, unless the entity is started from scratch. We agree with NRDC that this is unlikely to occur given the need for a speedy transition as well as the short contract terms offered to the administrator under the ORA/TURN Coalition proposal.

We also have no guarantees under the WEM/SESCO Coalition proposal that bidders will provide the statewide coverage for energy efficiency activities that are needed to meet the Commission's energy efficiency goals. As discussed above, the choice of what measures to install (and where) is left to the competitive market, based on the geographic region or market(s) that the various competing administrators propose to cover and the assessment of costs and risks by implementers competing to operate under standard offer contracts. While the proponents of this approach consider competition and de-centralized decision making to be an advantage, we see clear disadvantages to relying exclusively on this administrative model to meet our aggressive energy savings goals in California. We note that standard offer programs have never been used to support energy appliances or building standards, which are cornerstones of California's energy policies.

The experience in Texas to date reinforces our concerns. While supporters of the WEM/SESCO Coalition proposal contend that it is a great success in Texas, we need to view that success in proper perspective. In particular, the goals of the program are extremely modest by California standards. The Texas statutes require that distribution-only IOUs in that state achieve savings equivalent to only 10% of the electric utility's annual growth in demand by January 1, 2004, and each year thereafter. The standard offer programs in Texas, which is the only type of energy efficiency program offered by the IOUs in that state, met this goal for 2004 by contracting for approximately 150 megawatts (MWs) and 400 gigawatt hours in savings. To put this amount in perspective, 150 MWs represents the equivalent in usage for roughly 38,000 homes in Texas.⁹⁷ California's energy goals for energy efficiency will require a much more aggressive and comprehensive commitment to deferring or displacing more costly supply-side alternatives.

We are not persuaded that reliance on a standard offer program, as the WEM/SESCO Coalition envisions, can achieve these goals. Under this model, statewide programs (other than a 5% proposed budget for education and information) could cease to exist entirely, as there is no apparent coordination between the multiple administrators and no assurance that each administrator would select the statewide program for its service territory. In fact, the administrative model proposed by the WEM/SESCO Coalition would make coordinating different programs very difficult, if not impossible. Customers would be faced with multiple and sometimes overlapping programs that invite customers to participate in more than one program intended to achieve the same energy savings. Having an uncoordinated group of implementers design and implement a variety of uncoordinated programs will not achieve the program synergies and leveraging necessary to optimize savings from energy efficiency, as required by the Energy Action Plan.

Moreover, experience to date also suggests that standard offers may not be well suited to tapping the full potential of cost-effective energy efficiency, particularly large commercial and industrial installations. According to staff at the Texas Public Utilities Commission, coverage of those sectors has been lacking because standard offers may not provide the flexibility needed for the more complicated, but large savings potential, applications.⁹⁸ We believe that the WEM/SESCO Coalition proposal is even less likely to produce better results in this regard, since it relies exclusively on the "deemed savings" standard offer. In Texas, this type of standard offer is considered suitable only for limited applications, primarily residential and small commercial. More elaborate approaches to verifying program savings, including metering, billing analysis or computer simulation, are required under the Texas standard offers for most other applications.⁹⁹ While we can support the general concept of performance-based contracting, we do not believe that the "one size fits all" version of this approach proposed by the WEM/SESCO Coalition is appropriate for California. As an implementation strategy, standard offer programs may have a place; however, it is not a strategy that can wholly replace other, more appropriate, administrative models.¹⁰⁰

Based on the above, we find that the administrative proposals recommended by ORA/TURN Coalition and WEM/SESCO Coalition would introduce significant start-up costs, uncertainty and delays in the future administration of energy efficiency programs. Moreover, by relying on a competitive RFP process for the selection of administrators, these approaches introduce the additional risk that the solicitations will not produce administrators with the requisite experience and capability to manage California's large energy efficiency program and meets the goals of the Energy Action Plan. As discussed above, the WEM/SESCO Coalition proposal magnifies this risk by delegating the role of program choice to individual implementers competing among themselves to operate under standard offer contracts.

In contrast, returning the IOUs to a lead role in program choice and portfolio management will not create the legal risks described above or require statutory changes. Transitioning from staff to IOU responsibilities would involve a relatively short transition period, and could be accomplished at an orderly pace that would not disrupt program delivery. Based on our experience with utility administration during the pre-restructuring (Collaborative) era, we are confident that the IOUs have the requisite expertise and capability to
administer energy efficiency consistent with the Energy Action Plan and the savings goals we establish in this proceeding. That experience has demonstrated
to us that IOUs can meet aggressive savings goals under an administrative structure that holds them directly accountable for program results. As we reported in D.03-10-057, we estimate that IOU administrators during the Collaborative produced $1.4 billion in net benefits to ratepayers (savings minus costs, including shareholder incentives) for programs implemented or initiated over the 1994-1997 period.¹⁰¹

In their comments, several parties argue that IOU administration brings with it inflated administrative costs and other inefficiencies that justify placing a different entity or entities in the Program Choice and Portfolio Management Role.¹⁰² We believe that these arguments could equally apply to any administrative structure in which administrative costs and overall program and portfolio performance were not adequately and accurately monitored. They raise a broader issue, namely, how to ensure that the program results, both costs and benefits, being reported by the IOU administrators, IOU implementers and non-IOU implementers are credible, particularly for resource planning purposes. We concur with Rich Sedano of the Regulatory Assistance Project that the specifics of who performs the program choice or portfolio management function are not relevant to this question. Rather, what is relevant is the structure of the monitoring and verification program, or what we refer to as EM&V.¹⁰³ We discuss that structure in Section  5.3 below.

In their comments, the CCSF and WEM argue that an administrative structure that places the IOUs in the role of Program Choice and Portfolio Management is inconsistent with the statutory requirements of AB 117. We disagree. AB 117 added sections to the Public Utilities Codes that permit cities and counties that have registered with the Commission as "Community Choice Aggregators," to buy and sell electricity on behalf of utility customers in their jurisdictions.¹⁰⁴ AB 117 contained provisions, codified at Section 381.1(a) that required the Commission to establish procedures by which anybody, including CCAs, can apply to become administrators of energy efficiency programs established pursuant to Section 381. We have interpreted our decisions that allow CCAs and other third parties to apply for PGC funds as consistent with this requirement while at the same time recognizing that, as the procedures for allowing CCAs to begin serving customers evolve, we may need to revisit the issue.¹⁰⁵

WEM construes the requirement that any party be allowed to apply to become an administrator of energy efficiency programs as meaning that such entities must be allowed to assume the responsibility for portfolio selection and management.¹⁰⁶ The City of Berkeley expresses similar concerns.¹⁰⁷ WEM argues that none of the proposals, except the one put forth by the WEM/SESCO Coalition, complies with AB 117. We reiterate our interpretation of "administrator" for purposes of AB 117 as meaning "any entity implementing an energy efficiency program that is the subject of Section 382, which authorizes the expenditure of certain funds on energy efficiency programs."¹⁰⁸ We believe this is consistent with the competing interests articulated in Section 381.1 as well as the requirements for handling ratepayer money, as discussed above.

At the same time, we have recognized that "we may ultimately find that CCAs are appropriately independent agencies that should have considerable deference to use Section 381 funds" and have reserved broader issues about CCAs role and discretion for later determination.¹⁰⁹ We are currently establishing the procedures required by AB 117 before CCAs begin serving customers, including obligations of CCAs, recovery of IOU costs, and required reports to the legislature.¹¹⁰ Once those details are resolved, we may revisit the issue of allocating electric energy efficiency PGC funds to CCAs in the context of their role in delivering electricity to their customers. Stated another way, we may revisit the question of whether CCA customers should be relieved of their responsibility for energy efficiency PGC and procurement surcharges if the CCA elects to take over these functions. Nothing in this decision prevents us from modifying the process for allocating PGC funds to CCAs in the future.

CCSF argues that there will be no meaningful opportunity for other implementers to compete under an administrative model that puts the IOUs in the role of program selection. It therefore follows, in CCSF's view, that the IOUs Coalition and NRDC/LIF Coalition proposals are inconsistent with AB 117's requirements.¹¹¹ As discussed in this decision, we believe that with quality control measures, the IOUs can both select and sponsor programs without bias.

During oral argument and in its subsequent reply brief, WEM argues that using the CPUC's definition of administrator to mean "implementer," Pub. Util. Code § 381.1(a) would require that the Commission retain the program choice function. ¹¹² According to WEM, "the statute requires the Commission to `weigh the benefits of the proposed program,' not delegate the weighing to other entities." ¹¹³ We believe that today's decision, which holds the IOUs responsible for assembling a portfolio of programs pursuant to the Commission's overall policy guidelines and energy savings goals, and for submitting recommendations to the Commission for ultimate approval, is entirely consistent with the language of § 381.1(a). Nothing in today's decision is intended to delegate that ultimate approval to the IOU administrators. We note that even when Energy Division staff has performed the program choice function by selecting program proposals using the Commission-adopted criteria, the Commission has approved or disapproved the recommendations of staff.

In conclusion, we find that returning the IOUs to the lead role in program choice and portfolio management will best meet California's goals for integrated resource procurement, for several reasons. First, this approach is the most effective way to hold the IOUs accountable for the responsibilities they have been assigned by both the Legislature and this Commission to procure demand-side and supply-side resources to meet Energy Action Plan goals. It is also the logical corollary to the market structure we have established in R.01-10-024 for procuring supply-side resources. In addition, in contrast to proposals for independent administration, it is an approach that can be put in place without new statutory authority, and without substantial start-up costs, uncertainty or delays.

At the same time, we realize that returning IOUs to these roles will also require us to institute appropriate safeguards, as part of our overall approach to quality control. We discuss those safeguards in the following section.

5.2. Quality Control Measures For Program Choice and Portfolio Management

Irrespective of what entity or entities fulfill the role of Program Choice and Portfolio Management, we need to adopt quality control measures to ensure that program administrators select programs and manage them in a manner that is consistent with our objectives. We believe that the measures discussed below, in combination with our regulatory and policy oversight, are necessary to ensure that IOUs integrate an optimal mix of energy efficiency programs into their resource plans as they perform these functions.

Before turning to that discussion, we first respond to a related issue raised by the TURN/ORA Coalition. In its April 8, 2004 filing, the TURN/ORA Coalition recommends that the Commission reject the notion of adopting performance incentives to motivate the performance of administrators-irrespective of whether the administrator is the IOU or a third-party.¹¹⁴ In its April 26 comments, NRDC argues that at least some part of the administrator's earnings should be based on performance-no matter who the administrator-to ensure that ratepayers get the most for their investments in energy efficiency and do not pay no matter what the level of performance.¹¹⁵ We concur with NRDC and others that we need to consider a risk/reward mechanism for energy efficiency program administration in this proceeding. As indicated in prior rulings and decisions, we intend to do so in careful coordination with the development of an overall procurement incentive framework:

"In D.02-10-062, we expressed our preference to adopt a uniform incentive mechanism to provide an opportunity for utilities to balance risk and reward in the long-term procurement process...The goal of this effort is to motivate the IOUs to procure least-cost supply-side resources and make cost-effective demand-side investments, taking into account the environmental costs (or benefits) of various resource option. Our challenge will be to create an overall procurement incentive framework that aligns the interests of utility investors, management and ratepayers such that the proper balancing of these preferred resources occurs in the procurement of power from existing and new resources."

"As discussed in D.03-12-062 and D.04-10-050, any incentive mechanisms being considered in resource-specific proceedings (e.g., energy efficiency) must be consistent with the overall procurement incentive framework we adopt in this proceeding. [footnote omitted] Accordingly, we intend to adopt an overall framework for procurement incentives before we make our final determinations on resource-specific incentive mechanisms. Nonetheless, some work on resource-specific mechanisms may proceed concurrently, since several key aspects of those mechanisms (e.g., performance basis and measurement protocols for energy efficiency) will need to be developed irrespective of the overall procurement incentive structure. We will also consider, on a case-by-case basis, issuing interim decisions in resource-specific proceedings on aspects of incentive design, as long as doing so will not prejudge our determinations in this proceeding."¹¹⁶

Accordingly, we will address the issue of a risk/reward mechanism for IOU administrators during a subsequent phase of this proceeding, consistent with the direction in R.04-04-003. With or without an incentive mechanism, however, we recognize the need to adopt measures to address potential bias in the program selection and portfolio management process. For this purpose, we adopt a minimum requirement for competitive bidding and an advisory group structure for the Program Choice and Portfolio Management functions. In addition, we ban all affiliate transactions between the IOUs and program implementers, as discussed in Section 5.2.3 below.

5.2.1. Competitive Solicitations.

With regard to program selection, we believe that competitive solicitations can provide an important safeguard against bias in that process. Most importantly, competitive solicitations can help to identify innovative approaches or technologies for meeting savings goals with improved program performance that might not otherwise be identified during the program planning process. As the NRDC/LIF Coalition points out, however, not all program activities lend themselves to a competitive solicitation. It would be counterproductive to require open bids in instances where, for example, partnerships between IOUs and local governments can take advantage of the unique strengths that both partners bring to the table, or a combination of partnerships and bilateral contracting arrangements can deliver effective statewide initiatives, such as a statewide public awareness campaign or an upstream lighting program.¹¹⁷

As discussed above, the Energy Action Plan has placed energy efficiency back at the forefront of resource procurement activities in California. All program implementers-IOU and non-IOU alike-need to be selected and evaluated based on their ability to best meet our resource procurement goals, including the specific savings targets we establish in this proceeding. We also note that a large portion of IOU-implemented programs are already delivered through non-IOU third parties. For example, data provided by Energy Division indicates that PG&E and SCE each contract out approximately 40% of the energy

efficiency program dollars that they implement to non-IOUs via both competitive bidding and sole source contracts. Thus, it appears that non-IOUs are already actively involved in program implementation. In our view, decisions on whether non-IOUs should be program implementers responsible for designing and delivering the program (rather than working to implement IOU-designed programs) should be made based on an evaluation of whether the program designs and delivery mechanisms proposed by non-IOUs are superior to those currently being implemented or planned for the future.

In other words, competition in energy efficiency procurement should focus on soliciting good, new program ideas to achieve (or exceed) Commission goals, rather than allocating a specific percentage of program funding to particular implementers. We will accomplish this in the future as follows: For each program planning cycle, the IOUs will propose a portfolio of programs (with input from the advisory groups described below) that reflects the continuation of successful IOU and non-IOU implemented programs and new program initiatives designed to meet or exceed the Commission's savings goals with cost-effective energy efficiency.

As part of that process, the IOUs will identify a minimum of 20% of funding for the entire portfolio that will be put out to competitive bid to third parties for the purpose of soliciting innovative ideas and proposals for improved portfolio performance. With input from the advisory groups described below, the IOUs will specify the portion(s) of the portfolio to put out to bid (for example, they could be sector-specific, could focus on peak savings, etc.), as well as the proposed bid evaluation criteria. The portions to put out to bid could encompass programs currently designed and delivered by a combination of IOU and non-IOU program implementers. The bid solicitation should be designed to improve performance of the portfolio in terms of producing the most cost-effective energy savings that meet or exceed our savings goals. Any current program or group of programs (IOU or non-IOU designed and implemented) that can be improved upon in this way may be subject to open bids to replace, augment or otherwise enhance current efforts. However, as discussed above, open bids should not be required in instances where current or potential future partnerships between IOUs and local governments can take advantage of the unique strengths that both partners bring to the table to deliver cost-effective energy efficiency services, or where combination of partnerships and bilateral contracting arrangements with private or public entities can deliver effective statewide initiatives that enhance portfolio performance. Such activities should be funded out of the 80% (maximum) core portfolio that is not put out to competitive bid. The proposed portfolio of programs, portions to put out to bid and the bid evaluation criteria will be filed by the IOUs in their program plan applications for each funding cycle, and subject to Commission approval. (See Section 5.2.4 below.)

To facilitate the review that will be necessary by the advisory groups described below, Energy Division (or its consultant(s)) should work with the IOUs to compile all administrative and non-administrative costs and energy savings data on current programs in a standard format that will facilitate direct comparisons across programs, and make any needed refinements to the existing reporting requirements to facilitate such a comparison. This information should be made available to the advisory groups by March 15, 2005. Updated cost and savings information in the standardized format should also be submitted with the IOUs program funding applications.

We believe that a 20% minimum requirement for open bidding along the lines discussed above captures the potential benefits of competition and serves as an added safeguard against selection bias. At the same time, it provides sufficient flexibility to avoid imposing competitive bidding on program offerings that are more effectively delivered using other approaches. We will adopt the 20% minimum rerquirement for the next funding cycle, beginning in 2006, but may modify it for subsequent funding cycles, as appropriate.

In its reply comments, LIF raises a concern that competitive bidding will result in "cream skimming."¹¹⁸ Cream skimming describes the situation in which only the lowest cost energy efficiency measures are designed and implemented, leaving behind other cost-effective opportunities for energy efficiency. We share LIF's concerns over cream skimming, with an important qualification. As we have stated in the past, the pursuit of the most cost-effective measures first does not, per se, constitute cream skimming. This approach only becomes a problem when lost opportunities are created in the process, that is, when long-lived, cost-effective savings are "lost irretrievably or rendered much more costly to achieve, if not exploited promptly."¹¹⁹ Thus, in the past we have directed IOUs to "pursue the most cost-effective DSM resource programs first, if doing so does not create lost opportunities."¹²⁰ This policy should apply equally to third-party bidders and non-IOU program implementers, and should be reflected in the bid solicitation and evaluation criteria.¹²¹

In terms of how to develop the RFPs, evaluate the bidders and make final selections, we will use procedures similar to the ones we recently adopted for supply- side competitive solicitations in D.04-01-050, utilizing the advisory group structure we adopt today. We discuss that structure and describe the portfolio design and selection process in further detail below.

5.2.2. Advisory Group Structure

Based on the proposals and comments in this proceeding, we are persuaded that advisory committees or working groups can also help to safeguard against the potential for bias in program selection and portfolio management. They can do so by: (1) promoting transparency in the program administrator's decision-making process; (2) providing a forum to obtain valuable technical expertise from stakeholders and non-market participants; (3) encouraging collaboration among stakeholders; and (4) creating an additional venue for public participation. As described in this decision, we have used advisory committees and working groups extensively in the past for this purpose on the demand-side, and have recently created advisory review groups in our Procurement Rulemaking to help safeguard against selection bias on the supply-side.

However, we have also encountered legal obstacles in those instances where Commission-appointed advisory boards have been directed to manage a portion of ratepayer collections, without prior statutory authorization. In addition, we have found that some advisory committees and groups in the past have been less effective than others in fulfilling their intended purpose. We have drawn from our experience, as well as from the thoughtful comments of the parties, to develop an advisory group structure for the Program Choice and Portfolio Management functions. In our view, the result is a structure that will provide the benefits discussed above while keeping the administrative structure manageable.

In the discussion that follows, we provide general guidance and expectations for the advisory group structure, but purposefully do not specify every implementation detail. The IOUs should put together the advisory groups and implement the program design and selection process consistent with today's decision in the spirit of the collaborative approach they discuss in their filings. The ALJ, in consultation with the Assigned Commissioner, may provide additional clarification and direction on these issues, as needed. We also require Energy Division to provide the Assigned Commissioner with a written assessment of the effectiveness of the advisory group structure we establish in this proceeding, on an annual basis. Energy Division may conduct this assessment itself or hire an independent contractor for this purpose, whose costs will be paid for out of energy efficiency program funds.

For the Program Choice and Portfolio Management functions, we believe that advisory groups can be a valuable component of the administrative structure on two levels. On one level, they create the forum for an open and informative exchange of information among program administrators, industry experts and stakeholders as the IOUs develop their program selections for Commission consideration, and manage their program portfolio throughout the funding cycle. On another level, advisory groups can serve an important "peer review" function by providing an independent assessment of the IOUs' portfolio design and program selections. We believe both levels of advisory group input will be valuable under our adopted administrative structure.

To this end, we direct the IOUs to establish three "program advisory groups, or PAGs"¹²² drawing from the energy efficiency expertise of both market and non-market participants across the full spectrum of program areas and strategies. One PAG should be established for PG&E's service territory, one for SDG&E's service territory, and one for the combined service territories of SCE/SoCalGas. One purpose of these PAGs is to provide guidance to the IOUs regarding region-specific customer and program needs, and provide a forum for input and collaboration with the local interests and stakeholders served by the programs.

However, the PAGs must not focus exclusively on region-specific needs. In the interest of keeping the total number of advisory groups manageable, we do not specify the formation of a separate advisory group for the purpose of addressing statewide program design, selection and implementation. Nor do we establish a separate statewide group to advise us on open-ended policy and program management issues, such as the CEAC proposed by the Collaborating Parties. We agree with NAESCO that the advisory functions proposed for the CEAC appear overly broad and ambitious. We believe that the resolution of significant policy and program management issues can be better achieved through other procedural venues, including workshops, Commission-directed studies under the Analysis and Support function, as well as the solicitation of written comments from interested parties.

Nonetheless, we direct the IOUs and their PAGs to also address statewide programs and consistency issues, bringing in national expertise as appropriate to consider these issues. For this purpose, the IOUs should form a subgroup of their PAG members who will closely collaborate and coordinate on statewide programs that cut across IOU service territories. These include statewide marketing and outreach, support for building codes and standards, education and training and other activities that secure both short- and long-term energy savings and peak demand reductions by providing a consistent and recognizable program presence throughout the state. In addition, the PAGs and IOUs should collaborate on statewide program designs and implementation strategies that increasingly integrate energy efficiency with demand response and distributed generation offerings to end-users.

Moreover, we expect the IOUs and PAGs to ensure that statewide residential and nonresidential program offerings take advantage of "best available practices" and avoid customer confusion by being as uniform and consistent as possible. While we recognize that differences in climate zones and other parameters may warrant some variations in program offerings to customers, these variations should be the exception and not the rule. The IOUs are responsible for ensuring that the design of statewide programs proceeds in this coordinated manner. If the need emerges to focus on a particular market segment, the IOUs and PAGs may also establish a separate working group of industry experts and stakeholders to address that need. We prefer this approach to the Focus Plan presented by Cal-Ucons, which would create multiple Commission-appointed advisory groups for this purpose.¹²³ In this way, the portfolio design and program selection process can "stay current with the dynamics of the marketplace"¹²⁴ without creating an additional layer of advisory committees that would potentially be duplicative of the PAG membership and structure.

Energy Division and ORA staff will be ex officio members of each PAG and peer review subgroup (see below) and CEC staff is invited to participate as ex officio members as well. The IOUs will select additional PAG members, but participation will be voluntary and there will be no formal voting rules or designation of voting and non-voting members. Each PAG member will need to devote the time necessary to meet and confer with the IOUs on program design, selection and portfolio management, and provide written comments to the IOUs as appropriate.

On an annual basis, the PAGs will provide a joint report to the Energy Division with recommendations on how the IOUs can improve their effectiveness as administrators in managing the portfolio of programs, including how the program selection process could be improved to better meet the Commission's procurement goals. If consensus on these issues cannot be reached, the report should present consensus and nonconsensus positions. Consistent with our treatment of procurement review groups on the supply-side, those parties eligible to receive intervenor compensation awards in this proceeding should be eligible to seek compensation for their work as PAG members.¹²⁵

PAG members will provide advice and feedback to the IOUs and provide annual information to the Commission, but will not have any independent decision-making or contracting authority. The IOUs are expected to work closely with the PAGs throughout each program cycle, meeting with them at least quarterly. We do not expect that all input from the PAGs will necessarily be agreed to by the IOUs (or even among PAG members), but it is our hope that the forum for introducing new ideas and identifying problems specific to the IOUs' proposals will narrow the scope of differences considerably. PAG members (including those on the peer review subgroups described below) do not, in any event, relinquish their rights to participate in energy efficiency proceedings and comment on IOU filings in those proceedings.

We believe that the PAG meetings should be open to the public,¹²⁶ and the IOUs should establish a clearinghouse website for noticing these meetings and posting documents to be discussed by PAG at the meetings. However, these meetings are intended to facilitate discussion and exchange between PAG members and the IOUs. Accordingly, the IOUs should establish appropriate protocols for obtaining comment from public participants during those meetings, e.g., they may designate a specific amount of time at the end of the meeting to take comments or questions from the "floor." In addition, the IOUs are expected to conduct public workshops at least twice a year that are designed to solicit broad public input from non-PAG members concerning program design and implementation.

The IOUs are required to provide advisory group members with comprehensive information on program implementation activities and proposed program changes, and take other steps to ensure that members have an opportunity to review the information and work with them to improve program implementation. It is the IOUs' responsibility to provide notice for all advisory group meetings and public workshops, arrange for meeting space and conference call dial-in numbers, reproduce and distribute meeting materials and provide other administrative support to the advisory groups (and subgroups described below), as needed.

Within each PAG, the IOU will also identify and select a subgroup of non-financially interested members with extensive energy efficiency expertise that are willing to serve as peer reviewers in this process, along the lines of the Independent Observer described under the initial NRDC/LIF Coalition proposal. We refer to these subgroups as "Peer Review Groups, or PRGs." On the supply-side, advisory group membership was put together by a combination of the IOU (1) sending out notices to non-market participants it identified on the service list and (2) taking requests from parties asking to be including in the membership. The IOUs may utilize these and other informal approaches for selecting members for each PAG and its PRG subgroup of non-financially interested peer reviewers. The IOU administrators should notify the Assigned Commissioner and ALJ by letter of the individuals that are selected to serve on the PAGs and PRGs.

We define a non-financially interested member for the purpose of serving on these PRGs as follows:

A financially interested party is any person who engages in the purchase, sale or be marketing of energy efficiency products or services, or who is employed by a private, municipal, state or federal entity that engages in the purchase, sale or marketing energy efficiency products or services, or who provides consulting services regarding the purchase, sale or marketing of energy efficiency products or services, or an employee of a trade association comprised of entities that engage in the purchase, sale or marketing or energy efficiency products or services.

Energy efficiency services include among other things, performing energy audits and advising clients and potential customers about potential energy savings they can achieve, but does not include evaluating, measuring and verifying the installation and/or results of energy efficiency products or services, or research to develop new energy efficiency products or services.

Members of each PRG will be expected to participate in the ongoing PAG process. In addition, they will be expected to review the IOUs' submittals to the Commission and assess the IOUs' overall portfolio plans, their plans for bidding out pieces of the portfolio per the minimum bidding requirement, the bid evaluation criteria utilized by the IOUs and their application of that criteria in selecting third-party programs. The three PRGs are also expected to meet and assess the statewide portfolio (represented by the combination of the four IOUs separate portfolios) in terms of its ability to meet or exceed short and long-term savings goals in compliance with the Commission's policy rules. Energy Division will chair the PRGs, and also take an active role in the PRG process on a substantive level.¹²⁷ Energy Division may hire an independent consultant or consultants to assist in its own assessments of these issues, which will be paid for out of energy efficiency program funds. The IOUs will be required to include the PRG assessments with their Commission filings for approval of program plans and final programs.

Finally, in order to ensure that the advisory groups are organized and managed in the most effective fashion, we recommend that the IOUs define a charter or mission statement for the PAGs and PRGs and ask them to review, discuss and adopt the mission in their own words. These mission statements should include sunset provisions. The IOUs should also assign contact person(s) in their organizations for the advisory groups, and clearly define the nature of written or oral produces expected. We also suggest that the sponsoring IOU meet with the chairs of these advisory groups and Energy Division on a regular basis to discuss how the process can be improved. Above all, we emphasize that advisory group members must be treated with respect and courtesy.

5.2.3. Affiliate Transactions

In response to opening comments, the TURN/ORA Coalition clarifies its position that, under the "Efficiency California" proposal, an organization that has a parent company, affiliate or subsidiary implementing energy efficiency programs in California would not be eligible to serve as the program administrator.¹²⁸ This raises the issue of how affiliate transactions should be handled in the context of today's adopted structure for energy efficiency, i.e., with the IOUs in the Program Choice and Portfolio Management roles. The Assigned Commissioner solicited further comment on this issue in her October 7, 2004 ruling.

ORA, TURN, CCSF and Cal-Ucons urge the Commission to completely ban affiliate transactions for energy efficiency implementers, arguing that the potential for cross-subsidies between IOUs and their affiliates and anti-competitive conduct is no less present on the demand-side than it is on the supply-side, for which the Commission adopted a complete ban on affiliate transactions in early 2004. SESCO and WEM specifically argue that no IOU or affiliate effort should be allowed within the 20% competitive minimum bid requirement.

The IOUs, on the other hand, argue that prohibiting affiliates from competing to provide energy efficiency services is unwarranted because, by its open and competitive nature, the energy efficiency program market presents much less potential for self-dealing than sources of generation. Moreover, the IOUs contend that excluding utility affiliates could unfairly put additional pressure on them to meet already stringent energy savings goals. They posit that the affiliate transaction rules adopted by D.97-12-088 provide ratepayers with adequate protection against cross-subsidies and anti-competitive conduct in the context of the program select process envisioned by this decision.

We disagree for the reasons presented by TURN, ORA and CCSF:

"[W]hile the Commission has not yet adopted an incentive structure for energy efficiency program implementers, the utilities may well end up with a clear financial rationale for preferring their affiliates as program implementers. They may have the opportunity to earn profits directed from the programs they implement and indirectly from those implemented by their affiliates. The [decision's] strongest quality control mechanism is the advisory group structure. The Peer Review Groups in particular are intended to eliminate utility bias in the program selection process during the competitive solicitation. They are also supposed to monitor the utilities' portfolio design decisions, such as what portion to put out to bid and which partnerships to form. This is an enormous task to assign a group of volunteers assembled by the utilities, and it is a much larger task than that assigned to the supply side counterpart. The supply side Procurement Review Groups have significantly fewer bids to review than the volume the Peer Review Groups will face. Moreover, the Procurement Review Group workload tends to be spread through the year, whereas the Peer Review Groups will be asked to look for manifestations of bias or other problems in a lump sum of proposals." ¹²⁹

Moreover, we find the IOUs argument that a ban on affiliate transactions could limit their ability to meet our energy efficiency savings goals particularly disingenuous, given the fact that IOU affiliates currently provide a very small share of energy efficiency services relative to the IOUs themselves and non-IOU affiliated third parties. ¹³⁰ In fact, this is probably the best point in time to initiate such a ban, since it will enable us to avoid the types of problematic transactions that arose in the past with the emergence over the years of many new IOU affiliates under contractual relationships with the IOUs on the supply side.

For the reasons discussed above, we adopt an affiliate transaction ban between IOU administrators and program implementers for energy efficiency, without exception. This ban will apply to transactions between the IOU administrator and any implementer that is an affiliate of PG&E, SCE, SDG&E or SoCalGas. A broad ban on affiliate transactions is appropriate and reasonable in the context of energy efficiency. This is because many of our cornerstone energy efficiency efforts are designed to "transcend borders" with respect to individual IOU service territories, and provide energy efficiency services and resource savings on a statewide basis. Adopting a broad affiliate transactions ban at the outset avoids the need to revisit the issue of "who's affiliate" each time a program is deployed in more than one IOU service territory.

5.2.4. The Post-2005 Portfolio Design and Program Selection Process

Based on the above, we envision a portfolio design and selection process that will proceed as follows, beginning with the 2006 planning cycle:

1. The Commission establishes overall policy guidelines and energy savings targets for the development of post-2005 program portfolios.

2. With input from their Program Advisory Groups and the broader public, the IOUs design a comprehensive portfolio of programs to meet the long-term needs of their resource portfolios, including specific localized needs,¹³¹ and consistent with the Commission's overall policy guidelines and energy savings targets. The portfolio should reflect the continuation of successful IOU and non-IOU implemented programs and new program initiatives designed to meet or exceed these targets.

3. During this design process, the IOUs identify what components of the portfolio they plan to bid out to third parties (minimum 20%) for the purpose of soliciting innovative ideas and proposals for improved portfolio performance, and what criteria they propose to use in evaluating bids. The Peer Review Group reviews and provides feedback on the portion of the portfolio that the IOUs plan to put out to bid, as well as the evaluation criteria. As discussed in this decision, Energy Division staff may hire independent contractors to assist in providing feedback to the IOUs on these issues. The Peer Review Group assessment should also address whether the statewide portfolio meets the Commission's policy objectives.

4. At least two public workshops are held during this stage of the process to solicit broader public input on the design of the portfolio (including the bid selection criteria) before the IOUs' file their program planning applications for approval with the Commission. The applications will include a description of the portfolio composition, the pieces that will be put out to bid and the IOUs' proposed evaluation criteria. The assessments of the Peer Review Groups will be appended to the IOUs' filings. As described in this decision, the applications will also include the most up to date cost and savings information for all current programs, in a standardized format.

5. Upon receiving Commission approval of the applications, the IOUs complete the process of selecting programs and program implementers to design and deliver the programs described above. The utilities develop and issue RFPs using criteria approved by the Commission and, select a set of bids. The Peer Review Groups (including Energy Division independent consultant(s)) observe the IOUs' bid selection process to ensure that the criteria are applied properly.

6. Before finalizing their selections, the IOUs discuss the proposed results of their bid review process with the Peer Review Groups (and Energy Division's independent consultants). For this discussion, the IOUs will provide the program implementation plans, timelines and goals of the bidders in as much detail as available, along with any other bid evaluation information that the Peer Review Groups may request. This group will have an opportunity to ask questions about how the criteria were applied and provide feedback on the selection process, and otherwise help to ensure that the bid process is fair. It is the IOUs responsibility to describe in their compliance filing (see below) how they have responded to criticisms presented by the Peer Review Group (and Energy Division consultants) during this process.

7. After incorporating feedback, the IOUs make public all winning bids and submit compliance filings. If the Peer Review Group and IOU are in full agreement on the final program plans and bid selections, this filing will be made as an advice letter. If not, the IOU files a compliance application in the program planning application docket requesting Commission approval of the final programs. Written assessments of the Peer Review Groups (and Energy Division's consultants, as applicable) are appended to the compliance filings.

5.3. EM&V and Other Administrative Structure Issues

Today's decision goes a long way in creating our post-2005 administrative structure for energy efficiency by addressing the threshold issues related to Program Choice and Portfolio Management. We have focused on these issues today because they have been the most contentious. By resolving them we can now turn to other administrative structure issues and address those in the context of our adopted structure for those two key functions.

5.3.1. EM&V Administrative Structure

The most important issue remaining is what EM&V administrative structure we should put in place to: (1) develop EM&V policies, protocols and reporting requirements for Commission consideration; (2) contract for EM&V studies, and (3) assess the results of those studies. As described in Attachment 2, the recommendations in this proceeding on EM&V administrative structure share several similarities, but there are also differences with respect to what role advisory groups, program administrators and implementers should play in EM&V contract management. On the one hand, some parties argue that program administrators and implementers should have no role in contracting for or managing EM&V studies-even with contractors approved by the Commission. Other parties take the position that contracting by program administrators and implementers is appropriate, as long as it is coupled with an independent review process overseen by a measurement advisory group, Commission staff, or a combination of both.

We are persuaded by the comments of TURN, ORA, SDREO and others that the EM&V structure within the overall administrative framework must be free of conflicts of interest that could bias EM&V results. We think that TURN puts it best:

"EM&V serves both as the quality control mechanism for ratepayer funded energy efficiency programs and as a data input for IRP [Integrated Resource Planning] and the portfolio planning process. Rigorous, reliable EM&V is crucial to California's ability to attain its energy efficiency goals for three interrelated reasons.

"First, IRP requires independent EM&V. EM&V plays a determinative role in the reliability of energy efficiency savings. For energy efficiency to be considered a reliable resource in IRP, such that EE [energy efficiency] is taken seriously along side steel-in-the-ground resource by the IOU resource portfolio planners, California must have an EM&V framework designed to generate accurate and reliable data. Conflicts of interest that encourage compromised EM&V of programs jeopardize the success of IRP.

"Second, Independent EM&V ensures that ratepayers get the energy efficiency for which they pay. California needs an EM&V framework bold enough to prevent wasteful expenditures of ratepayer money on energy efficiency programs. Ratepayers should reap the benefits of the energy efficiency programs they fund. These ratepayer benefits should include well-run, effective energy efficiency programs, resultant lower customer bills, and increasing utility use of energy efficiency as a demand-side resource. Ratepayers deserve an administrative structure that gives them a reasonable assurance that their money is being wisely and efficiently expended.

"Finally, independent EM&V enables the program selector to assemble the strongest portfolio of programs. EM&V must be as transparent and independent as possible to ensure that the best program designs are adopted and that the best program implementers are selected. An EM&V structure that does not completely shield EM&V studies from potential conflicts of interests undercuts California's ability to reach our energy savings potential."¹³²

The EM&V administrative proposals presented by the IOUs Coalition, the NRDC/LIF Coalition and the Collaborating Parties fall short of ensuring the necessary independence of EM&V, in our judgment. Under the IOUs Coalition proposal, all program implementers contract directly with EM&V consultants for review of their programs, unless they opt to have the IOUs contract for EM&V in their place. The EM&V consultants are pre-approved as "independent" by the stakeholder EM&V advisory group, but as TURN points out, they nonetheless lose independence by entering into a direct financial relationship with the entity whose work they are to evaluate.¹³³

The original NRDC/LIF Coalition proposal assigns program level EM&V contracting responsibility to the IOU administrator, which results in independent EM&V for non-IOU programs only. The amended version of the NRDC/LIF Coalition proposal appears to adopt the approach advocated by the IOUs Coalition.¹³⁴ The Collaborating Parties do not agree on who should have contracting responsibility for program-related studies, but still assign the IOU administrator as the technical lead manager of those contracts. In our view, allowing the entity that selects the programs and manages the portfolio (IOUs) or
the program implementers (IOUs or non-IOUs) to manage or contract directly for EM&V of their own efforts could seriously undermine the independence of even the most conscientious EM&V consultants. Although ORA and other Collaborating Parties apparently believe that it is the contracting entity that exerts the most influence over the contractor, and not the lead technical contract manager, in our experience we have found that both roles can be influential on the progress and content of the study in question.¹³⁵ While we have allowed these arrangements for some studies in the past, we are persuaded by the arguments presented in this proceeding that we must improve upon our approach to EM&V, particularly in view of the renewed imperative that program results be credible and reliable for resource planning purposes.

At the same time, we recognize that the IOU Portfolio Managers need access to market information to perform their jobs of selecting and managing a portfolio of programs to meet the Commission's objectives. Similarly, program implementers need to access to information on a real-time basis to improve program delivery. Dictating that the only information they can receive is from studies managed and contracted for by others is counterproductive. Thus, we propose a process that allows the IOU Portfolio Managers and program implementers to manage a limited subset of evaluation studies as long as there is no potential for conflict due to the nature of the study, and as long as Energy Division has a lead role in the selection of contractors.

Accordingly, for the 2006 program year and beyond, Energy Division will assume management and contracting responsibilities for all EM&V studies that will be used to: (1) measure and verify energy and peak load savings for
individual programs, groups of programs and at the portfolio level (including load impacts, useful measure life, savings retention and persistence studies), (2) generate the data for savings estimates and cost-effectiveness inputs, (3) measure and evaluate the achievements of energy efficiency programs, groups of programs and/or the portfolio in terms of the "performance basis" established under Commission-adopted EM&V protocols and (4) evaluate whether program or portfolio goals are met. For purposes of this discussion, we refer to this category of studies as "program and portfolio impacts-related" studies.

More specifically, Energy Division will be responsible for: (1) allocating Commission-authorized funding for program and portfolio impacts-related EM&V among the individual studies, (2) developing the work scope for each study consistent with our adopted EM&V protocols, (3) writing RFPs and selecting the contractors, and (4) managing and contracting for the work. Consistent with the working relationships we have already established with the CEC in this proceeding, we anticipate that the CEC staff can be called upon to provide Energy Division with technical input and, if needed, staffing support for these functions.

Public participation in the development of these studies will be provided in several stages. First, the development of the EM&V protocols that determines the scope of these studies is being conducted via workshops in this proceeding, to be followed by further opportunity for public input via written comments before the Commission issues a final decision. Second, the overall EM&V plans, budget and the allocation of funding levels to program and portfolio impacts-related studies will be addressed during each program planning cycle, beginning with the 2006 cycle next summer. (See Section 6.) Third, as discussed further below, study results will be made available for public review and comment while in draft form, with a brief teleconference workshop scheduled as part of this process. Once the program and portfolio impacts-related studies are finalized under Energy Division's management, the studies will once again be made available for public review. The forum for review may be this rulemaking, a pending Commission proceeding (e.g., the AEAP) or a future Commission proceeding in which resource planning assumptions are being developed. The appropriate forum for this review will be established by Assigned Commissioner's ruling.

In carrying out the functions described above, Energy Division should utilize ad hoc review committees of technical experts, as appropriate. We recommend that Energy Division draw on the experience of the CEC with ad hoc committees under the Public Interest Energy Research (PIER) program in creating such committees for its own purposes. We also encourage Energy Division to take advantage of the EM&V expertise of IOUs and the field experience of implementers by inviting them to participate on these review committees as the issues may warrant.

We believe that a flexible, ad hoc technical committee approach is more valuable to the EM&V administrative structure we adopt today than the formal standing measurement advisory groups proposed by various parties. (See Attachment 2.) Review committees created on an as-needed basis can best provide Energy Division with flexibility in securing the EM&V technical expertise it requires for the particular task or issue at hand. This approach also creates much less of a burden on individual technical experts, who would be called on only as needed to review and provide input on a specific EM&V issue or work product. Based on the CEC's experience with the PIER ad hoc committees, we are confident that there exists a pool of EM&V experts in California and other states who are willing and able to periodically review written products and provide Energy Division with technical feedback at very little or no cost to ratepayers.

In addition to taking advantage of technical expertise through the formation of committees on an as-needed basis, as described above, Energy Division is directed to set up a team for each evaluation that includes the relevant implementers and administrator(s) and accomplishes the following objectives:

a. Implementers and administrators have an opportunity to provide input on study scope and priorities;

b. Implementers, administrators and evaluation contractors work together to develop data requirements to ensure a representative sample of customers and optimal data gathering, including a plan to ensure that the evaluator obtains all data needed for the evaluation in an expeditious manner; and

c. Implementers and administrators have opportunities to review and comment on evaluation methodology and results before the study results become final.

Providing the opportunity for public input into the initial study design process and opportunities for implementers, administrators and evaluators to share information and concerns during the evaluation process should go a long way toward minimizing the number of disputes that may need to be resolved. In addition, we agree with TURN, ORA, CCSF, NRDC and others that all stakeholders, including program implementers and administrators, should be given the opportunity to review and comment on each program and portfolio impacts-related study while in draft form. As proposed by members of the Reaching New Heights Coalition, a brief teleconference workshop should be scheduled as part of this process.

If disputes concerning the study findings remain after these informal review opportunities, the administrators, implementers or interested parties should seek Commission resolution. We intend to establish a set schedule for resolving all disputes for each program cycle as part of our EM&V protocols, which are being developed in a separate phase of this proceeding. In the meantime, we believe it is premature to adopt an automatic placeholder for alternative dispute resolution, as NRDC and other members of the Reaching New Heights Coalition recommend. Instead, the assigned ALJ will determine the need for evidentiary hearing or other approaches to dispute resolution on a case-by-case basis, as warranted by the factual basis of the complaint.

The EM&V administrative structure outlined above for studies that address program-related impacts will ensure a clear separation between "those that evaluate" and "those that do," i.e., IOU Portfolio Managers and both IOU and non-IOU implementers. However, we also acknowledge that there are EM&V studies that are designed to inform the Portfolio Manager about the overall performance of groups of program types working together, and that suggest changes in program design or mix as a result. There are also certain types of studies that provide program implementers with information needed on a real -time basis to improve program delivery. For example, process evaluations are undertaken to improve the design and efficacy of a particular program or set of programs while the programs are operating. "Best Practices" studies evaluate which energy efficiency programs or program features should be incorporated into future program designs. Other studies may be undertaken to review the effectiveness of training, audits or media campaigns. Still others may be designed to track efficiency "sales" for individual or groups of programs, or provide other accurate market information to help the Portfolio Manager and implementers fine-tune and improve energy efficiency procurement strategies. For this discussion, we refer to this second category of studies as "program design evaluation and market assessment."

Due to the focus and purpose of these studies, we believe that allowing the IOU Portfolio Managers or program implementers to directly contract for (and serve as technical lead in managing) program design evaluation and market assessment studies would not present a conflict of interest. The IOUs' performance will be evaluated based on their ability to meet the Commission's resource procurement goals for energy efficiency. It is in their best interest to objectively evaluate and fine-tune their portfolio as the programs are underway and obtain accurate market information for this purpose. Similarly, program implementers require some of this same information to enhance program delivery.

Moreover, it makes sense from a functional standpoint for the IOU Portfolio Managers to be responsible for managing studies that provide them with information needed for day-to-day management of the portfolio, for communicating timely feedback to their implementers and for improving portfolio performance over time. Having the IOU program administrators, and for some studies program implementers, manage these types of EM&V contracts also utilizes their in-house expertise in this area, thereby allowing us to focus Energy Division staff efforts on other priorities.

For similar reasons, we believe that the IOUs, rather than Energy Division staff, should take the lead in allocating Commission-authorized funding for this category of EM&V across individual studies, develop the scope of work for each study and prepare the RFP. The IOUs should solicit input from Energy Division, the CEC and program implementers during this process, and they may also continue to utilize CALMAC as a forum for obtaining technical input, at their option. As we have stated previously, CALMAC is not a Commission-created advisory group.¹³⁶ In any event, the IOUs must also provide opportunities for public input on the program design evaluation and market assessment studies as they are being developed and, once finalized, report the findings to the Commission and hold public meetings to discuss the findings of the studies. These opportunities for public input may be part of the public PAG meetings discussed above, or may be separately scheduled.

In addition, as discussed above, interested parties will also have an opportunity to participate in the development of the overall EM&V plans, funding levels and budget allocations across study categories during each program planning cycle. We envision a process whereby the Portfolio Managers and Energy Division, working with CEC and an ad hoc technical advisory group established for this purpose, develop a joint proposal on these issues. The joint proposal is then discussed in public workshops to obtain feedback before it is finalized. It is then submitted with the IOUs program plan applications during each planning cycle, for additional public review.

In their program plan applications, the IOUs should also describe each type of study (including general scope of work) they or their program implementers plan to manage and/or directly contract for under the program design evaluation and market assessment category. This will provide all interested parties a further opportunity to consider whether any of those proposed studies would, in their view, create a conflict of interest if the IOU Portfolio Managers or program implementers managed and directly contracted for them.

We believe that our adopted two-track approach to EM&V administrative structure balances the need to facilitate effective feedback to the IOU program administrators and program implementers, so that they can make mid-course changes to increase the effectiveness of the programs, with the need to protect against potential conflicts of interest. By splitting the responsibilities for EM&V administrative along the lines described above, this approach addresses two major concerns that various parties have raised. First, that an entity other than the one standing to profit from inflated program achievements should be responsible for substantiating program performance. Second, that Portfolio Managers and program implementers should be able to guide evaluation activities in order to make sure that those results can be used to improve the program and portfolio design while the programs are running, and when future portfolio and program plans are being developed.

However, there are two remaining issues we must address before being satisfied that this balance has been achieved. The first issue relates to potential conflicts of interest when EM&V consultants (or their firms) are also involved in energy efficiency program delivery, either as a subcontractor under IOU-implemented programs or as program implementers themselves. NAESCO, WEM, SESCO and Cal-Ucons urge us to address this issue by prohibiting entities from performing EM&V studies of any type at the same time they are under contract for program delivery work. In their view, the credibility of energy efficiency programs and associated savings would be compromised without establishing this strict "firewall" between implementers and evaluators. In particular, they argue that without such a policy, the IOUs could reward (or appear to reward) EM&V contractors for favorable study results by bestowing upon them contracts to implement energy efficiency programs. The IOUs, NRDC and other members of the Reaching New Heights Coalition, on the other hand, contend that creating such a firewall is less effective than requiring those parties bidding to become evaluators to fully disclose any potential conflicts, and for the Energy Division to use a transparent process to score bids on a case-by-case basis according to the degree of potential conflicts, as part of the overall bid evaluation process.

We believe that all parties commenting on this issue have raised relevant concerns for our consideration. We are persuaded that the two-track approach jointly presented by ORA, TURN and CCSF best addresses these concerns without compromising the principle guiding our choice of EM&V administrative structure, i.e., that of ensuring non-biased program evaluation results. Specifically, we will prohibit entities from performing any program and portfolio impacts-related studies at the same time they are under contract for program delivery work. As defined in this decision, these are the types of studies that are designed to produce findings (that may be favorable or unfavorable) on program or portfolio accomplishments. These findings, in turn, are likely to be considered in the program selection process during future funding cycles, may affect program payments to implementers, and may also result in program penalties or rewards depending upon the Commission's determinations on an energy efficiency risk/reward mechanism or overall procurement incentives. Therefore, a firewall between implementers and evaluators is most appropriate in this context. We note that other states have successfully used such a firewall approach, including Wisconsin, New York and Oregon. ¹³⁷

On the other hand, as TURN, ORA and CCSF point out, a firewall between program delivery and program evaluation is much less relevant for the subset of EM&V studies that we refer to as program design evaluation and market assessment. This is because these studies are designed to provide information feedback to administrators and implementers to improve program performance, rather than to produce findings related to program accomplishments. Therefore, excluding these types of studies from the firewall will mitigate some of the practical concerns raised by parties regarding a complete firewall, without undermining the EM&V structure.

We recognize that the limited firewall we adopt today may still force some market participants to limit their practices or reorganize their business structures. The Reaching New Heights Coalition raised concerns that this would result in fewer qualified firms bidding on evaluation projects, thereby compromising the quality of those studies. While we seek to soften the impact of the firewall by limiting its scope, we are persuaded by the arguments of TURN/ORA/CCSF that the quality of impact evaluation studies will not suffer as a result of a firewall. In particular, the experience and skills of individuals do not become lost while firms reorganize to meet market needs.

In fact, as TURN, ORA and CCSF point out, there is a well established pool of evaluation specialists to meet the demands of many other states where evaluators are prohibited from undertaking implementation activities.¹³⁸ Additionally, we are concerned by the prospect of a case-by-case review and scoring process related to potential conflicts for all types of evaluation studies
because of the impact on Energy Division. This type of review would be very burdensome to undertake, especially since the RFPs for most program evaluations will likely go out around the same time. Such case-by-case review should be the exception, rather than the norm.

In their comments on this issue, NRDC and other members of the Reaching New Heights Coalition lament that a firewall approach could unfairly foreclose many firms and contractors from doing evaluation even if they are engaged in unrelated implementation activities. One such situation they pose is that of a mechanical engineering professor at U.C. Berkeley who assists in the implementation of an air conditioning program. They raise the question of whether a statistics professor at U.C. Berkeley would then be foreclosed from assisting in the evaluation of that program. We believe that such individual circumstances can be effectively addressed, and have been in other states, without abandoning the firewall approach. For example, the Wisconsin Department of Administration has established a firewall between program delivery and evaluation "unless the Department of Administration determines that there is no conflict of interest" prior to commencement of the bidding process.¹³⁹ We leave it to Energy Division to develop bidder guidelines consistent with today's decision and to make the determinations it deems appropriate to address questions that may arise concerning the eligibility of a particular evaluator, based on the specific circumstances.

Finally, in recognition of the potential practical implications of any firewall, we adopt the TURN/ORA/CCSF proposal for a narrow exception to the firewall between implementers and program and portfolio impacts-related evaluators. We will permit Energy Division to lift the firewall and use a case-by-case review along the lines proposed by the Reaching New Heights Coalition only in the event that there are inadequate bidders for an evaluation project. Under such circumstances, Energy Division should issue a separate round of RFPs to include firms with potential conflicts. We believe that this is an unlikely scenario, however, given the well-established pool of evaluation experts both in-state and out-of-state. In all instances, with or without the firewall, bidders will be required to provide full disclosure of any potential conflicts of interests, including conflicts that may arise from non-implementation utility work.

We also clarify, in response to comments, that structurally separate non-IOU affiliates can separately perform EM&V and program implementation work without violating the firewall prohibition if they sign non-disclosure agreements that forbid contacts between the structurally separate firms on the contracted matters.¹⁴⁰ This serves to provide a clear demarcation as to what either business entity has been hired for and creates an effective firewall between the affiliated companies when performing work. We also clarify that the firewall will begin with a "clean slate" in the 2006 program cycle so that all current implementers and evaluators of the 2004-2005 programs may choose which option to pursue
for the 2006 program cycle.

Although we believe that the pool of eligible consultants for EM&V studies will not be significantly affected by the firewall we establish today, we do recognize that many of the same EM&V contractors that perform program design
evaluations and market assessments are the same ones that conduct program and portfolio impacts-related studies. This raises an additional concern, namely, that even the most conscientious EM&V consultants may feel pressured to "tread lightly" in presenting the results of program and portfolio impacts-related evaluations, knowing that the IOU Portfolio Managers (and program implementers) will be selecting contractors for other evaluation studies.

To address this concern, we will require that Energy Division make the final selection of any contractors hired by the IOUs or program implementers to perform program design evaluation and market assessment studies. For this purpose, we require Energy Division to solicit input from an ad hoc technical committee that includes the IOU Portfolio Manager(s) and program implementers that will be contracting for the study.¹⁴¹ Energy Division may structure the committee in any way that it believes will best enable it to make an independent determination of the most qualified bidder.

We recognize that under most circumstances the managing and contracting entity would also select the contractor. However, we believe that this exception is warranted and necessary given the circumstances. If the IOU Portfolio Managers (or program implementers) decline to manage and contract for program design evaluation and market studies for which they do not select the contractor, then Energy Division will assume responsibility for managing and contracting for these studies as well. In any event, the IOUs are required to promptly pay the contractor invoices for all EM&V studies managed by Energy Division, upon approval of those invoices by the staff contract manager. All EM&V studies will continue to be funded through PGC collections.

Today's decision provides clear direction on the EM&V administrative structure and process for developing EM&V program plans and budgets that we envision for the future, beginning with the 2006 program planning cycle. Nonetheless, we recognize that some of the implementation details will need to be further delineated, such as more specific lists and definitions for the studies that fall under the two EM&V study categories, as well as for the research and analysis activities discussed in the following section. To this end, Energy Division and CEC should jointly develop an implementation roadmap for EM&V and Research and Analysis consistent with today's decision, and hold public workshops to solicit input from the IOU Portfolio Managers, program implementers and other interested parties prior to finalizing that document.

The assigned ALJ will issue the roadmap for the 2006 program planning cycle by ruling, after considering written responses of interested parties to the Energy Division/CEC final proposal. The ALJ make provide additional clarification and direction on these issues or make modifications to the roadmap during the program planning cycle, as needed.

5.3.2. Research and Analysis in Support of Policy Oversight

With respect to Research and Analysis in support of our policy oversight responsibilities, we believe that the best approach is to have our Energy Division take the lead in this area as specific needs arise. As described in Attachment 1, this involves: (1) performing research and developing recommendations to assist in developing energy efficiency policy goals and priorities, program performance goals and funding levels, (2) evaluating the remaining potential to achieve additional energy or peak savings in both the short- and long term, and (3) performing other research, as needed, related to procurement and PGC funded activities.

Examples of activities that fall under this area of responsibility include: savings potential studies, assessments of net-to-gross ratios and updates to the Database for Energy Efficiency Resources (DEER). We place these activities under the management of regulatory staff because they involve judgments that can influence either the development of performance targets or the measurement of program achievements. For example, in both DEER and net-to-gross ratio work, judgments need to be made about what specific energy savings numbers from which studies will be used to estimate energy savings for specific measures. Due to the conflict-of-interest concerns discussed above, the IOU Portfolio Managers would not be the appropriate entities to manage or directly contract for this type of work.

We will also explore creating a more formal arrangement with the CEC for collaboration in this area and in EM&V, building on the working relationship we have established in this proceeding. For this purpose, we direct our Executive Director to contact his counterpart at the CEC with the goal of developing of an interagency memorandum of understanding (MOU) for CEC staff participation in EM&V and Research and Analysis in Support of Policy Oversight. In addition, we will continue to collaborate with the CEC at both the staff and Commissioner level on a broad range of energy efficiency issues, as we have during this proceeding under Commissioner Kennedy's and CEC Commissioner Art Rosenfeld's leadership.

We decline, however, to involving one or more policy advisory groups in this area of responsibility on a standing basis, as some parties propose. We find this approach to be far more structured and potentially cumbersome than we believe is necessary. In performing the Research and Analysis functions, Commission and CEC staff should have full flexibility to obtain input from various sources, including working groups of experts or hired consultants, as they deem appropriate to the circumstances.

5.3.3. Quality Assurance and Policy Oversight

As part of our overall Quality Assurance and Policy Oversight responsibilities, we will perform a number of functions necessary to ensure that program results are accurate and that ratepayer funds are being spent and managed in a responsible and productive manner. The discussion that follows highlights some of those functions, but is by no means exhaustive of our oversight authority.

The Energy Efficiency Policy Manual 2, and subsequent versions of the manual and related policy rules as adopted by the Commission, will continue to govern the development, management and evaluation of ratepayer-funded energy efficiency programs.¹⁴² As indicated in Section 6 below, one of the first priorities for the coming months is to update those policy rules.

EM&V protocols and procedures, cost-effectiveness methodologies and energy savings assumptions will continue to be approved and prescribed by this Commission. In some instances, we may establish a process for reviewing and approving case-by-case exceptions to these parameters that does not require a formal Commission decision. However, we expect that Commission staff would have a lead role in that process.

As needed, financial audits of the IOU administrators and their program implementers (IOU or non-IOU) will be conducted by Commission staff, or by consultants under contract to the Commission. The Commission will determine and prescribe what action is to be taken by the IOUs in response to audit findings and recommendations. Energy Division will also continue to be responsible for monitoring administrative and other costs, at both the program and portfolio-level, and for recommending adjustments as appropriate for Commission consideration. We may establish the frequency of these financial audits and cost monitoring reports as part of the 2006 program planning process, by Assigned Commissioner's ruling, or by other means, as appropriate.

The design and oversight of program-specific, portfolio-level and financial reporting requirements will remain the responsibility of the Energy Division. In order to assist Energy Division in fulfilling this responsibility, in a previous decision we authorized the expenditure of PGC funds for the creation of a data management system, now known as the Energy Efficiency Groupware Application (EEGA). The purpose of this system is to ensure that energy efficiency reporting is organized, accurate, consistent and useful to the public, the Commission and others charged with ensuring that California's energy needs are met.

As part of each program planning cycle, we require that the IOUs continue to reserve a portion of energy efficiency funding for the purpose of maintaining and expanding EEGA, as it will be a valuable tool for both the Commission and the IOUs in monitoring and assessing program and portfolio performance. The IOU Portfolio Managers will be responsible for reporting portfolio and program-level information using the standardized reporting formats, definitions, timelines and narratives established by the Energy Division, as updated from time to time. We will use these reporting formats to track savings, cost-effectiveness results and to support our resource planning and goal setting activities.

The IOUs should present proposed funding for EEGA as a separate budget line item in its 2006 and subsequent program planning applications. Energy Division, or its contractor, will perform the work to maintain and expand EEGA, and the IOUs will submit the necessary portfolio and program-level data in the format and frequency that Energy Division requires. The IOUs should forward all program implementation plans to Energy Division, as they are received, along with any other program or portfolio data that Energy Division may require in order to monitor program performance. Energy Division will be responsible for determining the final scope of work for any maintenance and enhancements of EEGA. In the interest of time, Energy Division may choose to use the existing contracting structure for EEGA.

In addition, we may continue to require a standard contract for agreements entered into by the IOUs with their own contractors or with non-IOU program implementers. As part of the 2006 program planning process, the IOUs (in consultation with the PAG) should address whether the current or modified versions of standard contract agreements with non-IOU contractors and implementers should be retained.

Finally, nothing in today's decision prohibits Energy Division from monitoring and evaluating the scope and quality of inspections done by the IOUs on their own programs or those they implement by contract. Moreover, identifying deficiencies in IOU program administration, and taking appropriate actions to rectify those deficiencies, will continue to be a function of this Commission and our staff.

In their comments on the draft decision, several parties urge us to take immediate steps to ensure that Energy Division is adequately staffed to address the energy efficiency responsibilities required by today's decision. As discussed in this decision, we will continue to work collaboratively with the CEC to utilize their staff expertise in energy efficiency, as appropriate, and have also authorized Energy Division to hire consultants for EM&V and PRG review functions. However, we agree with ORA and others that reallocation or augmentation of Commission staff resources to work on energy efficiency matters is likely to be required in order to fulfill the Energy Division responsibilities described in today's decision. We therefore direct our Executive Director to address these energy efficiency staffing matters with the management team and reallocate or augment staff resources on energy efficiency, as needed, without delay.

¹⁰⁷ City of Berkeley Opening Comments, p. 4.

http://www.cpuc.ca.gov/static/industry/electric/energy+efficiency/rulemaking/resource4.pdf