28. IT Costs
PG&E utilized its formalized PG&E Delivery Methods (PDM) process to assess and develop the IT functionality needed to meet its business stakeholders' requirements. As described in PG&E's testimony, the business requirements were developed by the stakeholders and communicated to the IT team responsible for this project. IT then began its own assessment of the IT changes that would be required, including an iterative process with the stakeholders to assess the impacts of the business requirements on the IT systems. To assist this process, the responsible business stakeholders facilitated the tracking and refinement of these new requirements and tracked the changes made. Through this process, the business requirements were more specifically defined. Once this process was completed, the business users formally approved the final set of high-level business requirements, so that IT could then further distill the IT requirements. These requirements and costs were then broken down into specific workstreams and deliverables for the project.
Through this PDM planning process, PG&E identified three areas of work that needed to be completed as part of Dynamic Pricing Phase 1: billing system changes; CSOL changes; and the CC&B version upgrade to Version 2.2. PG&E indicates that it has devised a detailed plan to build the necessary rates needed for PDP, build the customer facing CSOL tools and capabilities for PDP and ultimately for RTP, and re-platform CC&B in time for RTP.
28.1. Billing System Changes
PG&E bills its customers through two primary systems. The Alternate Billing Service is used to bill approximately 20% of PG&E's revenue; that encompasses about 20,000 monthly bills to mainly large C&I customers. CC&B bills PG&E's other customers, and thus, represents the majority of PG&E's billing activity measured by monthly bills issued as well as by revenue. In order to implement new tariffs in these billing systems, PG&E states that it must (1) first update its billing systems to update the newly implemented rate schedules, including functionalities such as calculating bill protection amounts for different classes of customers and reservation capacity for others; (2) add new PDP-related adjustment types to its billing systems that allow for things such as the reversal of charges and cancel/re-bill of the associated PDP rates; and (3) update the interfaces to and from dependent and related systems to recognize the PDP rates. PG&E indicates that it will also need to calculate unique adjustments to the bills sent to individual customers based upon these new rates, which must meet Revenue Reporting and Reporting Solution System reporting requirements associated with CPP and bill protection amounts.
PG&E estimates these billing system changes, excluding contingency, will cost $27,454,313 ($25,939,290 in capital for 2009 - 2010 and $1,515,023 in expense for 2008 - 2009) to implement. No party opposes PG&E's proposed billing system modification activities or the associated cost estimates. They are reasonable and the cost estimates will be adopted and included in determining the revenue requirement for this proceeding. Contingencies are discussed further on in this decision.
28.2. CSOL Update Changes
PG&E proposes to upgrade its CSOL systems for PDP implementation. According to the company, the Commission has made it clear that it wants tools and features that customers, faced with new dynamic tariffs, can use to effectively make informed decisions regarding their electric usage and applicable tariffs. PG&E describes two co-dependent improvements that it is making to its CSOL systems -- the improvement of the tools and functionalities, and the CSOL re-platform. PG&E estimated these costs to be $23.3 million assuming it simultaneously re-platformed its CSOL system. Included in this amount are re-platforming costs of $10.7 million that reflect a foundation re-platform estimated at $7.4 million and a Middleware re-platform to BEA Systems (BEA) estimated at $3.3 million.
With respect to the re-platform, PG&E states that it made this decision after performing a cost comparison analysis and determining that it was less expensive to both build the tools and re-platform than to build the tools without re-platforming. Also, according to PG&E, the tools will better satisfy customer needs and expectations if built on a new platform.
PG&E presented the costs to provide new PDP tools and functionality on its website and other functionality that the Commission wanted, such as multilingual support and "My Account" upgrades. No party has disputed the need for this scope of work or PG&E's estimated costs for it.
As part of the CSOL work, PG&E determined that it would be cost neutral, if not less expensive, to re-platform CSOL at the same time that it upgraded the tools. Further, PG&E determined that this plan would lower the implementation risk and increase the quality of the tools and other functionalities provided to customers. By conducting the CSOL re-platform now, PG&E indicates that it can provide the performance needed to service the higher transaction load that will be a result of PDP's more complex rates, rate features, and tools. This higher level of performance would allow PG&E to build tools that are more useful to its customers, thus furthering the Commission's goal of having appropriate tools to assist customers with making informed choices. According to PG&E, the old CSOL platform would not be adequate to deliver this same level of customer experience, and no intervener group provided evidence to the contrary.
According to DRA, review of PG&E's material shows that the approximate $10.7 million of PG&E's forecast related to re-platforming is not incremental in that it is not necessary to meet the requirements of D.08-07-045, has no support beyond the roughest of estimates, and/or reflects costs for replacement of previously funded IT efforts which show premature inadequacy.
DRA's position on the CSOL rewrite is that it will need to be done at some point in the future, but the evidence does not show that the CSOL rewrite must be done to support D.08-07-045. According to DRA, PG&E has presented forecasts of web traffic that show up to only a 25% increase in average volume, and only 90% increase in peak volume, adding that these forecasts are based on PG&E's overly optimistic forecasts of meter deployment which, in turn, dictates the number of eligible customers. In light of this and PG&E's own testimony, DRA asserts that PG&E can build the needed tools for default customers while deferring the $10.7 million cost for re-platforming.
DRA indicates that the key driver for re-platforming seems to be PG&E's desire to capitalize on the opportunity to upgrade its website to improve customer capabilities, prepare for future needs and improve quality of service, and that DRA recognizes the improvements that can accrue from the proposed change to WebLogic and BEA middleware. However, it is DRA's position that PG&E has not made a compelling case nor defined the specific functionality commensurate with the costs that justify authorizing recovery of those costs in this proceeding.
Furthermore, DRA argues that PG&E's desire to construct a new website IT architecture reflects poor IT planning that has resulted in performance decay. DRA indicates that the stress imposition from the PDP decision will add complexity and volume, but the scale of change-outs proposed by PG&E results from presently inadequate systems with deficiencies that are highlighted by the lack of current functionalities necessary to carry out PDP. According to DRA, PG&E's assumptions relating to cost neutrality, call times, need for scalability and the increased computing volumes are speculative and without support. DRA, therefore, recommends that the Commission order PG&E to continue to develop necessary functionality, but defer the web re-platforming until it has been vetted through the GRC process.
DRA states that PG&E's assertion that it would be cost neutral, if not less expensive, to re-platform CSOL at the same time it upgraded the tools, is not supported by credible evidence. According to DRA, PG&E has exaggerated the user interface updating costs for not re-platforming. Specifically, the claim that the older platform requires about $7.8 million more to add the same incremental functionality, which translates to almost 6500 man-days of effort (at $1200/day) or 32 man-years of effort, is unreasonable, and PG&E uses unsubstantiated costs that bias the analysis.
We will adopt the elements and costs33 of PG&E's CSOL upgrade proposal including that for re-platforming. Our understanding of the re-platforming issue is that while PG&E and DRA agree that it should be done at some point and improvements can accrue from the change, PG&E would like to do it now in conjunction with updating CSOL functionality, while DRA prefers that it be deferred and considered in PG&E's next GRC. We further understand DRA's position that PG&E's analysis of cost-neutrality may be flawed and overstated. However, DRA has not provided an independent analysis of how much more, or less, it would cost to implement the same functionality on the current system rather than on the re-platformed system. In addition, agricultural customers have requested the ability to navigate the PG&E website with a single login to access multiple meter data. This can only be accommodated on the re-platformed system. This is also apparently true for CSOL functionality that will be needed for RTP, which is scheduled to be offered as soon as May 1, 2011.
We note that DRA raised concerns regarding (1) website traffic volumes that do not justify high forecast costs, (2) difficulty in reconciling the gap between the functional capabilities mandated under the AMI previous rulemaking and those required to support PDP, (3) inadequate vetting of the current or future capabilities inherent in the previously selected AMI related systems and applications, and (4) CSOL functionality costs that far exceed simple incremental costs. We are however convinced by PG&E's responses which indicate that (1) the current site performs reasonably for today's needs, but PDP adds complexity that the current site cannot support; (2) with AMI, a customer had access to historical information to see static usage with SmartRate, whereas with PDP customers will need the ability to analyze what-if scenarios for the future; (3) the current meter data management system is already running at capacity and cannot handle more functionality; and (4) enhanced CSOL functionality will cause greater web traffic than what was contemplated in the AMI proceeding, not only due to the increased number of visitors but also because CSOL has to perform more tasks per visit.
Based on the above considerations, we determine that it is reasonable for PG&E to perform the CSOL re-platforming in conjunction with updating the CSOL functionality for PDP purposes. We believe this will minimize the ultimate CSOL costs to ratepayers while providing improved CSOL capabilities for a longer period of time.
28.3. CC&B Version 2.2 Upgrade
PG&E seeks ratepayer funding to transition to a new version of the Customer Information System, known as Customer Care and Billing, (CC&B). Specifically, PG&E seeks funding to upgrade from its current Version 1.5 to Version 2.2 at a cost of $31.3 million, before contingency, in this case. PG&E states that it presented the $31.3 million in costs for this upgrade in this Rate Design Window (RDW) filing because it recognized that CC&B would have to be upgraded ahead of the RTP implementation deadline. PG&E adds that a later version, Version 2.3, will be installed on top of Version 2.2 in order to support RTP, but is not yet available. PG&E states that it intends to seek additional ratepayer funding to upgrade again to Version 2.3 for an additional $28 million, before contingency, in its 2011 GRC.
PG&E identified four risks that it argues could be avoided by upgrading to CC&B Version 2.2 now and upgrading to Version 2.3 later. According to PG&E, the desire to mitigate these risks drove PG&E's decision to upgrade first to Version 2.2 and then to Version 2.3. The identified risks are:
1. Version 2.3 does not exist at this time;
2. PG&E needs to mitigate the risk of moving to Version 2.3 functionality and RTP by operating and stabilizing the re-platformed CC&B prior to making the extensive changes for RTP;
3. There is a risk that PG&E's IT department will be oversubscribing itself, doing one large Version 1.5 to Version 2.3 upgrade plus the RTP upgrade, all at the same time; and
4. Due to contractual terms, PG&E's Version 1.5 will not have vendor support after June 2011.
DRA originally proposed that PG&E focus efforts on CC&B Version 2.3 instead of Version 2.2 and implement only the necessary CC&B enhancements in the interim. DRA recommended that the Commission remove the $31.3 million requested by PG&E in this proceeding and instead consider it along with the Version 2.3 upgrade in PG&E's 2011 GRC.
However, DRA states that after submitting its testimony, it learned from the vendor, Oracle, that Version 2.3 would have been ready in September 2009 but for modifications PG&E asked Oracle to make, and Version 2.3 will be available in December. DRA asserts this gives PG&E approximately 14 months to perform the upgrade and meet the schedule set forth in D.08-07-045. Having discussed the Version 2.3 timeline directly with Oracle, DRA now recommends that the Commission authorize ratepayer funding of the entire $31.3 million for the CC&B upgrade, but direct PG&E to apply it to Version 2.3 and remove any Version 2.3 funding requests from its 2011 GRC.
If the Commission is inclined to allow funding for Version 2.2, then DRA argues the amount of that funding should be significantly reduced. DRA states that the evidence shows that PG&E's cost per customer for installation far exceeds the $2 per customer that Oracle suggests, and that even if Oracle's estimate is tripled to $6 per customer, there would still be a $28 million savings from PG&E's estimate. According to DRA, PG&E's estimates for related storage costs are either excessively high or for large storage volumes that PG&E has not shown it needs, noting that storage costs can range from 10 cents a gigabyte at Best Buy, to $250 a gigabyte for very expensive, very high quality production storage units. It is DRA's position that PG&E has not justified the estimate of $1 million for storage, and it should be rejected.
DRA states that although it has modified its recommendations relating to the CC&B Version upgrade, its reasons for disagreeing with PG&E's proposal remain the same. First, PG&E's claimed need for immediate replacement of Version 1.5 reflects questionable IT planning to date. In its 2005 AMI proceeding, PG&E requested approximately $66 million to upgrade from Version 1.3 to Version 1.5. In February 2009, PG&E filed this application seeking over $31 million for an upgrade to Version 2.2 and will have filed a GRC application in December 2009, seeking another upgrade, this time for more than $28 million, to Version 2.3. According to DRA, PG&E's ratepayer should not have to shoulder these costs which call into question PG&E's overall strategic IT planning and whether PG&E is making any effort to contain its IT costs.
A second and related DRA concern goes to the timing of PG&E's request and whether PG&E is using D.08-07-045 as a vehicle to justify IT infrastructure investment it either should have made before, or can defer until Version 2.3 is available. DRA states that by using D.08-07-045 as a vehicle, PG&E has selected possibly the most expensive way to address upgrading its CC&B system.
With respect to the PG&E claim that, due to contractual terms, PG&E's Version 1.5 will not have vendor support after June 2011, DRA states that considering that PG&E is working with Oracle to develop Version 2.3, it seems extremely unlikely that Oracle would refuse to work with PG&E to maintain its CC&B system while the Version 2.3 upgrade is performed. In any case, DRA asserts that Version 1.5 licenses can be renegotiated or PG&E can obtain IT from third parties in the 12 to 14 months it takes to perform the upgrade to a new version of the software.
In its comments on the Administrative Law Judge (ALJ) proposed decision (PD), PG&E revised its CC&B upgrade request.34 According to PG&E, in December 2009, Oracle released CC&B Version 2.3, which was a number of months before its previously published release date. PG&E now asserts that it is feasible and preferable to upgrade directly from Version 1.5 to Version 2.3. PG&E states it will perform the Version 2.3 upgrade in 2010 and the first quarter of 2011.
In light of its agreement with DRA to go directly to Version 2.3, PG&E requests that the Commission approve the estimated upgrade cost of $31.3 million as reasonable and only make recovery of upgrade costs above $31.3 million subject to reasonableness review. PG&E argues there is no dispute now over the $31.3 million upgrade cost estimate, since DRA proposed to allow recovery in this case of that amount and to remove the Version 2.3 costs from the GRC. PG&E states that, if the Commission approves $31.3 million as a reasonable cost estimate for the upgrade to Version 2.3 and allows recorded costs above that amount to be recovered after reasonableness review, PG&E would remove the $8 million capital estimate for the Version 2.3 upgrade from the GRC.
In its reply comments on the PD, DRA indicates that it can agree with PG&E to include $31.3 million for the Version 2.3 upgrade in this application provided that all funding associated with the upgrade is removed from the GRC.35 DRA further indicates that care must be taken to adjust the GRC rate base and depreciation estimates to reflect the removal of the $8 million and also to properly reflect in the GRC whatever capital costs are authorized in this Rate Design Window proceeding. DRA states that if the final decision in this case includes the $31.3 million for the CC&B upgrade, then it would work with PG&E to make sure the correct adjustments are made in the GRC.
Also, DRA does not oppose PG&E's request that only upgrade costs in excess of $31.3 million should be subject to reasonableness review.
It is now clear that it is reasonable to directly upgrade PG&E's CC&B from Version 1.5 to Version 2.3. Also, PG&E and DRA agree that the costs that should be included in this proceeding for that upgrade should be $31.3 million. It is not clear whether PG&E now believes this is a reasonable estimate of the cost for the upgrade or that it is merely a reasonable amount to include at this time, with the provision that any likely excess costs can be recovered through a separate reasonableness review. It is also not clear if DRA independently estimated this amount for the Version 1.5 to Version 2.3 upgrade or merely used PG&E's Version 2.2 upgrade estimate as its estimate in consideration of its position that PG&E's combined estimate of approximately $66 million for transitioning first to Version 2.2 and then to Version 2.3 was excessive. In any event, $31.3 million appears to be a reasonable amount to include in rates at this time. As indicated, that is the amount that PG&E originally estimated it would need to upgrade from Version 1.5 to Version 2.2. It does not include approximately $10.4 million in contingencies estimated by PG&E or the additional $8 million that PG&E has requested in its test year 2011 GRC to upgrade from Version 2.2 to Version 2.3. Actual costs may well be higher than this authorized amount. If so, it is reasonable to authorize recovery of those additional costs provided that PG&E can demonstrate such costs were reasonably incurred.
Therefore, as now agreed to by PG&E and DRA, we will adopt an estimate of $31.3 million in this proceeding for PG&E to upgrade CC&B Version 1.5 to Version 2.3, and order that PG&E remove all CC&B upgrade related costs from its test year 2011 GRC. If it chooses to do so, PG&E can request actual upgrade costs in excess of $31.3 million in a separate reasonableness review application. PG&E should file such application within 120 days of completing the transition to CC&B Version 2.3. Only those amounts found reasonable as a result of the Commission's review of that application will be reflected in rates on an ongoing basis. Any costs in excess of $31.3 million that are not included in the reasonableness review application shall not be included in rates on an ongoing basis. Also, the ratemaking treatment for recording PDP costs into the DPMA should be extended beyond 2010 to provide recovery through the DRAM of the revenue requirement associated with the CC&B upgrade costs.
33 Adopted costs exclude contingencies. Contingencies are discussed further on in this decision.
34 See Comments of Pacific Gas and Electric Company on Proposed Decision of ALJ Fukutome, dated January 11, 2010 at 2-4.
35 See Reply Comments of the Division of Ratepayer Advocates, dated January 19, 2010 at 1 - 2.
