VI. Comments on Draft Decision
The draft decision of ALJ Reed in this matter was mailed to the parties in accordance with Pub. Util. Code § 311(g)(1) and Rule 77.7 of the Rules of Practice and Procedure. Comments were filed on ____________ and reply comments were filed on_______________.
1. Performance measurements have been adopted in D.01-05-087.
2. Performance assessment criteria have been adopted in D.01-01-037.
3. The FCC has strongly encouraged states to establish regulatory incentives to ensure that ILEC OSS performance does not present barriers to competition.
4. The FCC has stated that RBOC Section 271 applications must be in the public interest to be approved.
5. The FCC has stated that "the fact that a BOC will be subject to performance monitoring and enforcement mechanisms would constitute probative evidence that the BOC will continue to meet its section 271 obligations and that its entry would be consistent with the public interest."
6. Since the initial filing of this proceeding, the parties have collaborated to establish performance measures, performance assessment criteria, and incentive payment structures.
7. The Administrative Law Judge convened a three-day workshop to develop a payment structure that would determine monetary amounts (performance incentives) paid by the ILEC for deficient OSS performance.
8. Pacific, Verizon, the CLECs, and ORA submitted performance incentive payment structure plan proposals.
9. Pacific and Verizon performed data runs on the submitted plans to assess the payment amounts generated by actual and simulated performance.
10. The payment amounts generated by Pacific, Verizon, the CLECs, and ORA's plans vary widely, ranging from approximately $50,000 per month for Pacific's plan to approximately $9 million per month for the CLEC's plan when the plans are projected onto Pacific's performance for the last quarter of 2000.
11. At parity performance levels simulated by Pacific, the payments range from approximately $10,000 per month for Pacific's plan to over $3 million per month for the CLECs' plan.
12. At non-parity performance levels simulated by Pacific that result in a 38 percent failure rate, the payments range from approximately $1 million per month for ORA's plan to over $48 million per month for the CLEC's plan.
13. Pacific's and the CLECs' plans propose a maximum annual liability at risk of thirty-six percent of Pacific's annual net return from local exchange service.
14. Pacific's net return from local exchange service in 2000 was $1,527,942,000.
15. Pacific's proposed maximum annual liability at risk is currently $550,059,120.
16. Pacific's plan's payments per performance failure are increased depending on the pervasiveness of performance failures, also termed the failure rate.
17. Pacific's plan proposes that Pacific be forgiven for up to the percentage of failures that would be expected under parity conditions except for the worst ten percent of the time.
18. Pacific's plan increases payment amounts for repeated failures.
19. Pacific's plan applies the 0.20 conditional critical alpha level to aggregate monthly samples larger than 30 cases.
20. Pacific's 0.20 conditional critical alpha level is applied only to three-month consecutive failures.
21. The CLECs' plan increases payments for repeated failures.
22. The CLECs' plan increases payments for the severity of the individual failures effectively using the statistical test p-value as a surrogate for severity.
23. The CLEC's plan forgives a maximum of fifteen percent performance failures, except that severe failures are excluded from the forgiveness plan.
24. The CLECs' 0.20 conditional critical alpha level is applied to sample sizes of less than 30 cases.
25. The CLEC's conditional alpha provisions include a decreased critical alpha level of 0.05 percent for aggregate samples.
26. Verizon's plan proposes a maximum annual liability at risk rising from approximately $20 million in year one to $40 million in year three.
27. Thirty-six percent of Verizon's 2000 net return from local exchange service was approximately $166 million.
28. Verizon's plan payment amounts are based on transaction volumes, generally the number of CLEC customers who experience service worse than the average level for Verizon's retail customers.
29. Verizon's plan payment amounts are based on a severity measure, the percentage of CLEC customers who experience service worse than the average level for Verizon's retail customers.
30. Verizon's plan proposes a 0.20 conditional critical alpha level, the same as Pacific's conditional alpha provision.
31. Verizon's plan has a forgiveness provision similar to Pacific's.
32. Verizon's plan leaves out performance measures required by D.01-05-087 and agreements between the parties.
33. ORA's plan proposes no payment caps.
34. ORA's plan would have the payments go the ratepayers.
35. ORA's plan does not forgive any identified failures.
36. ORA's plan increases payments for the severity of the individual failures effectively using the statistical test p-value as a surrogate for severity.
37. ORA's plan does not specify a 0.20 conditional critical alpha level.
38. A payment cap of thirty-six percent of annual net return from local exchange service has been adopted by four of the seven states with Section 271 approval, and the two other states have adopted similar percentages.
39. The FCC has approved a payment cap of thirty-six percent of annual net return from local exchange service as being a sufficient incentive to motivate non-discriminatory OSS behavior, in conjunction with other incentives.
40. Procedural caps are necessary to protect ILECs against unintended financial liability caused by unforeseen circumstances.
41. Monthly procedural caps payment amounts proportional to those adopted in New York and Texas are $15 million for Pacific and $4.5 million for Verizon.
42. The ILECs have only proposed provisions to reduce Type I error and not Type II error.
43. Proposed mitigation provisions decrease Type I error at the expense of Type II error.
44. Type II error disadvantages the CLECs.
45. The appropriate percentage of statistical failures that occurs from random variation has not been accurately estimated because it is affected to an undetermined degree by statistical artifacts and by the provision of better service.
46. Log transformations have not completely normalized average-based measure data.
47. The appropriate percentage of statistical failures that occurs from random variation can be calculated from accurate performance simulations.
48. The purpose of our incentive plan is not to reward or credit an ILEC for giving OSS advantages to the CLECs.
49. The purpose of our incentive plan is to ensure that an ILEC does not present OSS barriers to the CLECs.
50. A mitigation plan equal to or greater than the critical alpha level could serve as an incentive for gaming behavior.
51. If an ILEC provided ninety percent of its OSS service that was so good that random variation had been eliminated as a potential cause for missing a sub-measure, and the remaining ten percent of the service failed the performance statistical tests, it is most likely that nearly all of the ten percent missed performance measures are actual failures.
52. There is insufficient information in the record of this proceeding to appropriately apply a correction for random variation because each type of test will have a different failure rate at parity and non-parity levels.
53. The effect of a forgiveness percentage based on the critical alpha level would be arbitrary since critical alpha levels are selected without considering forgiveness percentage effects.
54. There is insufficient information in the record of this proceeding to determine the accuracy of the performance simulations.
55. Mitigation provisions are most important when an ILEC is providing parity OSS access.
56. It is unlikely that Pacific will provide complete parity within the six-month implementation period of our performance incentives plan.
57. The net resultant alpha level for Pacific's and Verizon's conditional alpha proposal is 0.008, much smaller than the unconditional standard, 0.10.
58. Pacific's and Verizon's conditional alpha proposals increase net resultant Type II error compared to the single-month application of the 0.10 alpha level.
59. Pacific's and Verizon's conditional alpha proposals reduce Type II error compared to using a 0.10 alpha level to assess each of the three months results for the Tier II chronic failure identification.
60. The application condition for the CLEC conditional alpha proposal is sample sizes of less than thirty.
61. Alpha level adjustments are helpful to decrease Type I error especially for large samples.
62. Pacific's assessment of the economic harm suffered by the CLECs from inequitable OSS access depends on multiple assumptions.
63. Changes in the assumptions in Pacific's assessment of economic harm from inequitable OSS access for CLECs cause large changes in economic harm.
64. Pacific estimates economic harm from thirty percent discriminatory service to be less than 0.04 percent of its net return from local exchange service.
65. Pacific offers payments equaling six percent of its local exchange service net return for thirty-eight percent performance failure rate.
66. The payment cap can provide a guide for setting payments for different failure rates.
67. The interpretation of lower failure rate outcomes is more ambiguous than the interpretation of higher failure rate outcomes.
68. A curvilinear relationship between the percentage of the payment cap and the percentage of performance failures can mitigate the ambiguity of lower failure rates if lower payment percentages are established for lower failure rates and payment percentages become increasingly higher as performance worsens.
69. Establishing a curvilinear payment guide that starts with a payment of from zero to one percent of the payment cap for service with a one to five percent failure rate adjusts for the ambiguity of lower failure rates.
70. Given the low power of the statistical tests ordered in D.01-01-037, it is likely that when two out of three statistical tests fail, the actual failure rate is closer to 100 percent.
71. Payments of 100 percent of the payment cap are warranted for identified failure rates of less than 100 percent.
72. Industry aggregate performance rates are generally about fifty-percent higher than CLEC-specific performance rates.
73. Establishing a curvilinear payment guide that reaches a payment of 100 percent of the payment cap for service with a fifty percent failure rate adjusts for small samples and low statistical test power.
74. Using the curvilinear payment guide for setting payments in relation to performance, Pacific's specific payment amounts are much less than the guide.
75. The payment amounts we select generally follow the curvilinear guide at lower levels than the guide.
76. Other changes to the performance incentives plan will likely increase the payment amounts from our estimates.
77. Because of the existence of many different variables that affect payment amounts and failure rates, comparisons with payment and failure rates in other states with Section 271 approval are not precise.
78. Holding the single-month alpha level constant for identifications requiring consecutive monthly failures produces a much lower net Type I error rate than the rate for the single-month assessment.
79. When the single-month critical alpha level (maximum Type I error) is 0.20, a statistical assessment requiring three consecutive month failures to be identified as a failure for the purposes of incentive payments has a net critical alpha level of 0.008 as calculated by the formula: p = 0.203.
80. When the single-month beta result is 0.30 (Type II error), a statistical assessment requiring three consecutive month failures to be identified as a failure for the purposes of incentive payments has a net beta result of 0.657 as calculated by the formula: p = 1 - (1 - 0.30)3.
81. When the single-month beta result is 0.30 (Type II error), a statistical assessment requiring six consecutive month failures to be identified as a failure for the purposes of incentive payments has a net beta result of 0.882 as calculated by the formula: p = 1 - (1 - 0.30)6.
82. A binomial calculation shows that requiring five out of six consecutive month results to fail a 0.20 critical alpha statistical test to identify a statistical failure for the purposes of incentive payments results in a 0.0016 net maximum alpha level.
83. A binomial calculation shows that when the single-month beta result is 0.30 (Type II error), a statistical assessment requiring five out of six consecutive month results to fail to be identified as a failure for the purposes of incentive payments has a net beta result of 0.58.
84. Requiring the higher payment levels for chronic failure identifications to continue for subsequent single-month failures until two consecutive months pass performance tests will reduce the potential for gaming behavior.
85. Requiring the higher payment levels for chronic failure identifications to continue for subsequent single-month failures until two consecutive months pass performance tests will increase the chances of identifying and correcting poor performance when it occurs.
86. The CLECs' and ORA's plans indirectly address severity by using the probability statistic, Z, as a surrogate for severity.
87. All other things being equal, as a performance failure becomes more severe, the corresponding Z-statistic becomes larger (smaller p-values).
88. A Z-statistic is also influenced by sample size.
89. A less severe performance result can have a larger Z-statistic than a much worse result if its sample size is sufficiently larger.
90. The CLEC and ORA severity proposals could identify one CLEC's less severe results as more severe than another CLEC's results even when this is not the case.
91. In general, Verizon's plan calculates the percentage of customers who receive service worse than the average ILEC customer (or the benchmark), and then uses that number as a measure of severity to adjust payment amounts.
92. The severity measure is an integral part of Verizon's transaction-based incentive payment system, and is difficult to convert to a sub-measure-based approach.
93. Pacific's proposal to apply statistical testing to benchmarks does not examine the effect of random variation on assessments with underlying non-compliant conditions.
94. Pacific's plan provides relatively consistent output and is correlated to aggregate failure rates for the year 2000.
95. The CLEC, Verizon, and ORA plans' payment amounts are either not significantly correlated to aggregate failure rates and/or are inconsistent month-to-month.
96. For Pacific's performance and payments, the correlations between payment amounts and failure rates are 0.42 for Pacific, 0.13 for the CLECs, -0.12 for Verizon, and -0.01 for ORA and only Pacific's correlation is significant at the 0.10 level (N = 12).
97. Pacific's plan payment amounts can be adjusted for Pacific and Verizon to account for the different size of the two companies and to match the "curvilinear" payment guide.
98. The CLEC plan payment amounts are much higher than our payment amount guide.
99. Verizon's and ORA's plans are inconsistent from month-to-month, producing wide variations in payment amounts that are not related to the relatively small variations in aggregate failure rates.
100. Other problems with severity and volume-related metrics make the Verizon, CLEC, and ORA plans difficult to implement consistent with the criteria established in this decision.
101. Several significant modifications are necessary for Pacific's plan to be consistent with important criteria.
102. Pacific, GTE, and the CLECs collaborated on 2000 GTE Workpaper #13, a list of performance measures and sub-measures to be excluded from the incentive payment plans.
103. Since our plan is scaled to Pacific's and Verizon's individual payment caps, their total payment amounts are no different than if fewer measures were used.
104. Where measures may be correlated in a performance incentive plan, there is still value in multiple measurements, unless the measures have perfect or near-perfect correlations.
105. There is no evidence in the record to suggest that the performance measures to be used in the incentive plan are so highly correlated that they add no value to the assessment.
106. The performance measures to be used in the incentive plan were established in a collaborative process.
107. To implement the performance incentive plan, the ILECs will need to implement monitoring, assessment, reporting, and payment provisions.
108. Inadequate CLEC forecasts of OSS demand would be cause for excluding incentive payments in the event that deficient OSS performance resulted from such forecasts.
109. The CLECs have agreed to provide forecasts as proposed by Pacific.
110. The CLECs and the ILECs are in the best position to know how to implement forecasts for the purposes of OSS operation.
111. In accordance with D.01-05-087, Pacific is required to report performance results by the twentieth calendar day of the month succeeding the reporting period.
112. Pacific proposes to make payments within thirty days of the due date of the performance results report.
113. Ratepayers are making a significant investment in the ILECs' OSS infrastructures.
114. To the extent that OSS performance presents competition barriers, the ratepayers will not benefit from their investment in the ILECs' OSS-related infrastructure and they will not have received the economic and social benefits of competition which motivated the 1996 Telecommunications Act.
115. Rule 33 and Tariff 38 billing surcharges are used to compensate Pacific and Verizon for the costs they incurred to implement local competition.
116. The Commission provides for surcredits to ratepayer in the event of poor service by a regulated telephone company.
117. Exogenous cost changes and other regulatory surcharges and surcredits are included in the annual Price Cap filings that Pacific and Verizon are required to make every October.
118. In the annual filings, the utilities identify specific cost changes (increases and decreases) that occurred in the prior period (e.g., from October 1 through September 30).
119. These cost changes are combined and summed to determine the dollar amount of surcredits or surcharges to be reflected on a customer's monthly bills during the next calendar year.
120. Surcredits and surcharges, such as Pacific's merger savings and local competition implementation costs, are distributed between three groups of services, IntraLATA Exchange, IntraLATA Toll Services, and IntraLATA Access Services, in proportion to each group's share of Pacific's total annual billing base.
121. The surcredit or surcharge percentages are applied to the tariffed rate of the individual services that comprise each of the three service groups (IntraLATA toll, access, and exchange).
122. The adopted surcharge or surcredit percentage is applied to the tariffed rate for the services in each service group and modifies the price that the customer pays for the respective service for the following year.
123. In D.00-09-037 and D.01-09-063 the Commission used Rule 33 and Tariff 38 as the mechanisms for the payment of Pacific's and Verizon's local competition implementation infrastructure costs by their customers.
124. Rule 33 and Tariff 38 surcharges/surcredits appear as separate line items on Pacific's and Verizon's bills respectively.
125. Using Rule 33/Tariff 38 mechanisms will delay payment disbursements to the ratepayers. For example, a payment incurred in January 2003 would not be reflected in the surcredits to be disbursed until 2004.
126. Since the line items have already been established, there is no need for the Commission to authorize the creation of new line items, thus avoiding billing system modification expenses.
127. There would be numerous logistical and efficiency problems in creating an entirely new structure to provide immediate payments to each individual ratepayer.
128. A monetary amount received in the future has less value to the recipient as the same amount received in the present.
129. A ratepayer should be "indifferent" to an amount received in the future versus an amount received now if the future amount were to be increased as if the ratepayer had spent or invested the money now.
130. Ratepayers should be "indifferent" to future payments if they perceive equity when comparing the interest rates they receive to the interest rates they pay to Pacific and Verizon.
131. Discrimination in restoring normal OSS services following widespread disruption due to accidents or other events could damage competition.
132. The record does not include an implementable EDR process.
133. A timeline for commencement of payments generated by new measures can be established in the performance measurement part of this proceeding.
134. Absence of ILEC liability for poor OSS performance to CLEC customers for the first three months of a CLEC's new service could jeopardize new competition.
1. Through this incentive plan, Pacific and Verizon should be subject to performance monitoring and enforcement mechanisms.
2. Procedural caps should be adopted to protect ILECs against unintended financial liability caused by unforeseen circumstances.
3. The selection of an appropriate forgiveness percentage would be arbitrary because it is dependent on the critical alpha level selected for other reasons.
4. As determined by the Commission-approved performance measures and assessments, Pacific is not providing OSS parity.
5. The CLEC conditional alpha proposal is consistent with our directions in D.01-01-037.
6. Our estimated payment amounts in California are roughly comparable to actual payment amounts in Texas and New York.
7. Information that indicates an increased Type II error likelihood will help target alpha level adjustments to decrease Type II error where it is likely to be more beneficial.
8. Information that indicates an increased Type I error likelihood will help target alpha level adjustments to decrease Type I error where it is likely to be more beneficial.
9. A reasonable "anchor" for assessing the full monthly payment cap amount is a single-month CLEC-specific failure rate of fifty percent.
10. Using the curvilinear payment guide for setting payments in relation to performance, Pacific's specific payment amounts are insufficient.
11. Adjustments for the severity of performance failures can enhance an incentive plan's ability to target the most deficient performance by making incentive payments greater for the more severe failures.
12. Statistical tests provide greater confidence (higher Z-statistics, lower p-values) when applied to larger samples, compared to otherwise equal small samples.
13. Without an examination of the effect of random variation on assessments with both underlying compliant and non-compliant conditions, we cannot fairly implement statistical testing for benchmarks.
14. A performance incentives plan should be consistent over time.
15. A performance incentives plan should reflect differences in performance.
16. A performance incentives plan should produce equitable outcomes for both ILECs.
17. Pacific's plan, with several significant modifications set forth in Appendix J, should be adopted as the best base plan consistent with important criteria.
18. The list of all the measures and sub-measures excluded from incentive payments, set forth in 2000 GTE Workpaper #13, should be adopted.
19. The CLECs should provide forecasts as proposed by Pacific in its March 23, 2001 proposed plan.
20. Pub. Util. Code § 2104 does not compel us to decree the incentive payments to be liquidated damages and the CLECs' exclusive remedy for discriminatory ILEC performance.
21. We have crafted this plan in concert with the parties in order to implement the federally mandated restructuring of the local market.
22. Pub. Util. Code § 454 gives the Commission statutory authority to establish rates and charges for regulated telecommunications companies.
23. The Commission should require Tier II performance incentive payments to go to ratepayers through Pacific's and Verizon's surcharge and surcredit mechanisms: Pacific's Rule 33 (Schedule Cal. P.U.C. No. A2.1.33 - Billing Surcharges of Pacific's tariffs), and Verizon's Tariff 38 (Schedule Cal. P.U.C. No. 38 - Billing Surcharges of Verizon's tariffs).
24. Since ratepayers are making a significant investment in the ILECs' OSS infrastructures, it follows that they should receive incentive payments, which are directly related to the extent that those infrastructures do not perform as they should.
25. Rule 33 and Tariff 38 billing surcharges are appropriately used to compensate Pacific and Verizon for the costs they incurred to implement local competition.
26. The Commission should provide surcredits to ratepayers in the event of poor service by a regulated telephone company.
27. The Commission should require the ILECs to make monthly payments into an interest-bearing memorandum account, with an interest rate equal to the tariffed rate the respective ILEC's charge their customers for late payment, with the interest compounded monthly, and with interest accrual beginning immediately after the incentive payments are due and continuing to accrue on all amounts not yet credited to the ratepayers.
28. The Commission should require that Pacific Bell and Verizon identify in their respective separated intrastate results of operations monitoring reports an adjustment clearly identifying the annual performance incentive payments, and remove from the California intrastate results of operations, and the earnings monitoring reports, the payments made to the performance incentive memorandum account.
29. Incentive payments should not be the exclusive remedy for deficient performance.
30. An implementable EDR process is not currently available for the incentives plan.
31. Until an EDR process is implemented, the ILECs should automatically make incentive payments as indicated by the incentive plan we adopt.
32. Until an EDR process is implemented, the parties should use currently available Commission procedures in any disputes regarding these payments.
33. When new measures are introduced, payments should not be made on performance failures until the fourth month.
34. Under the adopted incentive plan, results for the first three months with activity for a new measure should not be subject to payments.
35. Regardless of which day during the month a CLEC first accesses the newly measured OSS function, that month should be deemed the first month for calculation purposes under the adopted payment plan.
36. The first, second, and third months' performance results should not be subject to incentive payments, and the fourth month should be subject to payments, with the results reported on the 20th day of the fifth month, and payments due thirty days thereafter.
37. Delineated changes to the performance assessment requirements of D.01-01-037 should be made to successfully and efficiently implement the performance incentives plan.
38. The incentive plan set forth in Appendix J is reasonable, consistent with law, and in the public interest.
39. This decision should be effective today so that the incentive plan can be promptly implemented.
IT IS ORDERED that:
1. A performance incentives plan, which identifies performance failures and non-failures, as specified in Appendix J incorporated by reference herein, shall be adopted for Pacific Bell (Pacific) and Verizon California Inc. (Verizon ).
2. The performance incentives plan, comprised of the performance measurements adopted in Decision (D.) 01-05-087, the decision model adopted in D.01-01-037 and as modified herein, and an incentive payment component adopted herein, shall be implemented for an initial period of at least six months or until otherwise modified by this Commission.
3. The same performance incentives model shall be applied to Pacific and Verizon .
4. Incentive payments, as specified in Appendix J of this decision, shall commence the first full month following the effective date of this order.
5. Following the six-month initial period, the performance of the incentives plan model shall be reviewed. Such review shall examine how the incentives plan model is functioning and shall include any adjustments and modifications to the components as well as the resolution of any issues remaining from D.01-01-037.
6. The schedule for the incentives plan model review shall be set by separate ruling.
This order is effective today.
Dated ____________________, at San Francisco, California.
APPENDICES A THRU K
Appendices A thru K to R9710016, I9710017
