XI. Performance Measures for Batch Hot Cut (47 C.F.R. § 51.319(d)(2)(ii)(A)(2))

A. Background

The TRO states that in order to ensure that the ILECs' hot cut processes are being achieved, state commissions may establish performance measures to track the quality of provisioning, maintenance and repair of loops included in a batch hot cut process.²¹¹ While the TRO does not specifically require performance measurements with respect to batch hot cut migrations,²¹² the TRO does recognize the value of performance measures to track ILEC performance of their new batch hot cut processes. Performance measurements have been a contested topic in this proceeding.²¹³ Moreover, the Commission has an open proceeding (R.97-10-016) that requires measurements and incentive mechanisms for operations and support systems (OSS), including batch hot cuts, in order to ensure that ILECs OSS do not present barriers to CLECs' ability to compete.

B. Performance Measures Proposed For SBC Hot Cut Processes

The processes by which SBC completes both the CHC²¹⁴ and the non-coordinated FDT process, are subject to certain performance measures, as defined in the California OSS OII Performance Measurement Joint Partial Settlement Agreement (the JPSA),²¹⁵ for assessing the timeliness of loop conversions for CLECs.

Two pre-order queries are included as part of SBC's batch cut proposal. The IDLC identification tool allows the CLEC to check the migrating loop for the presence of IDLC facilities. A second new pre-order function, the Scheduler/Reservation tool, allows the CLEC to reserve a date and time for the loop migration activity. SBC agrees to diagnostically track the IDLC pre-check and reservation queries in Measure 1 to ensure the ongoing timeliness of these pre-order transactions.

Loop conversion completions where the CLEC has requested a scheduled date and time for migration are tracked by performance measures 9 and 9A. Performance Measure 9 (Coordinated Customer Conversion as a Percent On Time) tracks the percentage of CHC conversions completed on time. On time performance for this measure is defined as completion "by the committed time," which, for the purposes of this measure, means "within one hour of the committed order due time."²¹⁶ The approved standard for PM 9 for unbundled loop conversions is a benchmark of 95% on time.²¹⁷ Performance Measure 9A (Frame Due Time Conversions as a Percentage On Time) tracks the percentage of non-coordinated FDT conversion orders of up to19 basic UNE loops (with or without LNP), up to 19 DSL capable loops, and up to 99 telephone numbers (transitioned to the CLEC through the LNP process). Although SBC does not propose to modify these requirements, it has proposed separate diagnostic reporting in Measure 9 and Measure 9A of the various batch cut options to provide specific visibility to the operation of its BHC options.²¹⁸ SBC also recommends that the existing benchmark standards apply and that existing hot cut submeasures remain subject to incentive payments as defined in SBC's Performance Incentives Plan.

At the request of certain CLEC parties, the ALJ scheduled a collaborative workshop for parties to seek consensus on a comprehensive set of performance measure revisions that should apply to the hot cut processes of the ILECs. The workshop was held on March 22, 2004 with a subsequent series of conference calls. No substantive consensus was reached.

Even though participants did not reached agreement on many proposed changes to the measures, they have discussed and agreed upon some clarifying language in the measures' descriptions, methods of calculation and business rules. While these changes are not specifically required for the batch cutover options, parties agreed they were positive improvements to the design of the measures, and are included in Attachment 6 of SBC's opening brief and Attachment 4 of SBC's reply brief.

SBC recommends that any remaining issues concerning performance measures be presented to the JPSA participants. SBC also recommends that any such issues be given priority treatment and that they be resolved within 60 days of the commencement of the next JPSA review.

To the extent systems or process changes are implemented, SBC intends to perform its standard internal testing before commercial deployment, and anticipates that the batch process will be subject to performance measures that will ensure that SBC implements the process effectively. FCC Rule 319(d)(2)(ii)(A)(3) provides that a state commission "may require that the incumbent LEC comply with an average completion interval metric for provision of high volumes of loops." Until a new batch cut process is more fully defined, SBC argues that it is premature to determine whether a new "average completion interval" performance measure will be needed, but believes that modifications of the current performance measure applicable to the existing CHC hot cut process reflecting the new batch volumes and intervals will be sufficient.

The current Measure 9 assesses the timeliness with which SBC completes service conversions to CLECs that use the CHC process. Data are disaggregated by reference to Coordinated Conversions (excludes LNP) and LNP Conversions. SBC argues that they have been an effective means for measuring timeliness of order completion and the quality of the loops provisioned.

SBC recommends that all batch hot cut transactions be subject to performance incentives under the existing relevant performance measures and under the terms of the Performance Incentives Plan adopted by the Commission in 2002.²¹⁹ SBC argues that the incentives plan is the result of several years of negotiation and litigation between the parties and is designed to work in concert with the performance measures defined in the California JPSA. The Commission adopted the plan to ensure that SBC would provide nondiscriminatory OSS access to CLECs, and to ensure that SBC would not backslide on such commitments after receiving Section 271 relief to provide long distance.

C. Performance Measures Proposed For Verizon Batch Hot Cut Processes

Verizon believes that adequate performance measures are provided by only a limited number of modifications. Verizon proposes to modify current Measure 9 (coordinated conversions: hot cut) to recognize three categories of hot cuts and proposes certain changes to the "description" portion of the measure, specifically to recognize that IDLC loops are only covered by the Basic and Large Job processes. Because IDLC loops are already covered in the current hot cut process, Verizon believes that there is no need to have separate measurement and reporting of IDLC loops.

Verizon also proposes adding new products to measure 17 (percent troubles within seven days) and measure 42 (percentage of time interface available). Verizon argues that the CLECs proposal for hundreds of additional metrics are unworkable, unnecessary, and uneconomic.

Verizon does agree to certain additional modifications that were developed through collaborative discussions with MCI. Specifically, MCI and Verizon reached consensus that wire center, bi-monthly reporting by Verizon is not required for Performance Measure 9, recognizing that Verizon makes available wire center level reports through its new WPTS.

As part of this agreement, Verizon committed that the WPTS can provide the following capabilities, and that Verizon will support such capabilities indefinitely: (1) the ability for CLECs to query the status of all hot cuts for a particular Common Language Location Identifier (CLLI) code; (2) spreadsheet hot cut status reports that CLECs can print out; (3) access to historical data in WPTS for 100 days so near-term historical reports can be pulled by the CLEC; and (4) availability of "flat files" with hot cut data that can easily be downloaded to Excel spreadsheets so that data can be retained beyond 100 days by CLECs; and (5) the "flat files" must, at a minimum, include all the data needed to recalculate the metrics covering unbundled loop provisioning including the three types of hot cuts offered by Verizon. Should Verizon withdraw support for any of these capabilities at any time in the future, MCI reserves the right to request reports by wire center and on a semi-monthly basis.

Further, MCI agreed to withdraw various proposed performance measures in return for a robust measurement under PM 17. Verizon was not willing to agree to MCI's 2% benchmark, but believes that the standard in PM 17 should remain at parity. MCI claims that parity is not sufficient because Verizon doesn't provision hot cuts for itself, thus there is no reasonable retail analog to hot cuts for CLECs. Further, the 2% benchmark is applicable to Verizon's hot cuts in New York and to Bell South in Florida.

D. Performance Measures Proposed by the CLECs

Both AT&T and MCI argue that a comprehensive set of performance measures must be adopted to ensure that both SBC's and Verizon's hot cut processes are performing adequately throughout the hot cut process - from ordering through provisioning and maintenance.

MCI developed a set of proposed performance measures for the ILECs' proposed hot cut processes through the workshop process.²²⁰ MCI modified certain existing performance measures that have already been approved by the Commission as part of the ILEC/CLEC JPSA, and proposed new performance measures to track certain aspects of the ILECs' hot cut processes. MCI believes that its proposed performance measures provide an adequate tool for the carriers and the Commission to track and correct (if necessary) the ILECs' performance through the imposition of penalties. MCI characterizes its proposal as tentative, to the extent that the Commission has not yet finalized approval of the ILECs' proposed hot cut processes.²²¹

MCI's proposal for performance measures was set forth in Attachment 4 of its Opening Brief, with an updated revision in its reply brief. MCI recommends that modifications to existing performance measures be implemented by SBC within 90-120 days of the Commission's order in the TRO proceeding. For new measures, MCI recommends that SBC have the processes implemented within 150 days of the Commission's order, with the processes subject to testing at commercial volumes for some period of time. MCI proposes that performance measures track all ILEC activities, systems or processes that are new to, or changed by, the high volume of batch hot cuts arising if CLECs lose access to unbundled switching.

MCI argues that these performance measures should, at a minimum: (1) reflect the claims made by the ILECs regarding performance of batch hot cut processes; (2) track each of the hot cut processes on a disaggregated basis; (3) track performance for all types of migrations between and among carriers, including customer loops with voice plus data; (4) require sufficient frequency of data reporting to enable the parties to identify and measure any difficulties in the ILECs' hot cut process at an early stage; and (5) report on a sufficiently granular basis (e.g., wire center) so that carriers and the Commission can determine if any difficulties with the ILECs' hot cut processes are concentrated in certain locations or widespread.²²²

MCI argues that data reporting using the new levels of disaggregation and performance standards will also provide the correct incentives to SBC to resolve problems more timely. MCI also proposes that many of the measure changes be eligible for performance incentives to encourage SBC to come into compliance. MCI argues that simply modifying performance measures that track the wiring work related to hot cuts is inadequate, and that the JPSA should be modified to track the entire hot cut process. MCI set forth its proposed performance measures applicable to Verizon in Attachment 6 of its reply brief, reflecting agreements that MCI reached through discussions with Verizon subsequent to the March 22, 2004 collaborative, as outlined above.

AT&T participated in the batch cut performance metrics workshop on March 22, 2004, and in the first two calls thereafter. AT&T worked with other CLECs to attempt to reach agreement on performance measures. AT&T argues, however, that it is not possible to design performance measures for a process that has not yet been defined. AT&T proposes that once the batch cut processes are defined for SBC, the parties and Commission should work collaboratively to put performance measures into place as quickly as possible. AT&T proposes that the Performance Measure Collaborative continue with this proceeding and that Verizon not be given final approval of its BHC process until agreement is reached and Commission approval and implementation is completed concerning performance measures and incentives for Verizon's BHC process.

E. Discussion

We recognize that appropriate performance metrics are an important feature to evaluate whether SBC's and Verizon's hot cut processes are effective in meeting their intended purpose of providing a seamless, efficient, low cost means of cutting over customers from one switch to another in an environment where UNE-P is not available. Performance measures are necessary as an early warning system enabling CLECs to detect and ask ILECs to immediately correct any problems that arise throughout the end-to-end hot cut process. Performance measures provide an indication as to whether the ILECs' systems are able to correctly schedule and perform the actual work for the hot cut within a timely, efficient manner. Performance measures must be reported on a granular enough basis to allow CLECs to determine if there are particular central offices for which ILEC hot cut performance is deficient.

Given the highly technical nature of the performance measures, we believe that collaborative workshops should be used as at least the first step in developing appropriate measures. Although a collaborative workshop was held, full consensus was not reached concerning the appropriate set of measures to be applied in connection with hot cut processes to be approved in this proceeding. Continued consensus building is necessary before a comprehensive set of performance measures can be finalized applicable to the ILECs' hot cut processes. While we shall continue to coordinate with the JPSA process in
R.97-10-016/I.97-10-017, we shall require that the instant proceeding continue to be used for making ultimate determinations concerning performance measures and their adequacy as a basis to warrant approval of the proposed batch cut processes.

As a starting point, we approve the performance measure modifications in those areas to the extent agreement has been reached among participants. Specifically, we approve the clarifying language in the measures' descriptions, methods of calculation and business rules to which parties agreed, as summarized in Attachment 6 of SBC's opening brief. We also approve the proposed performance measure modifications offered by Verizon including those that were mutually agreed to between MCI and Verizon, as outlined above. Verizon shall remain responsible for the commitments to which it agreed concerning WPTS capabilities as part of the package of performance measures.

At this time, we shall not adopt the additional modifications proposed by MCI where no consensus was reached. We conclude that MCI has not justified as necessary all of its additional performance measures, particularly in view of the potential cost and complexity involved. On the other hand, we do not believe that a complete analysis of all the proposed performance measures has been completed, particularly in view of the ambitious schedule for this proceeding. Moreover, because parties did not know what final form the ILECs proposed processes would take, it was not possible to completely assess what performance measure revisions may be warranted.

We therefore direct the ALJ to schedule further workshop collaboratives for participants to consider whether additional performance metrics may be warranted for SBC and Verizon, taking into account the hot cut requirements that are adopted pursuant to this order that may not have been anticipated during earlier workshops. Also, to the extent that MCI or other parties believe that any additional performance measures are warranted, we shall require justification that such measures are necessary and cost-effective before approving them.

²¹¹ TRO, ¶ 489.

²¹² TRO ¶ 489.

²¹³ Exhibits 39, 143, and 154C.

²¹⁴ The SBC Coordinated Hot Cut is also known as a To Be Called Cut. For the sake of uniformity, SBC refers to the process as a Coordinated Hot Cut or CHC.

²¹⁵ The JPSA is the product of the ongoing Commission Rulemaking and Investigation (R.97-10-016/I.97-10-017) into Monitoring Performance of Operations Support Systems Investigation (Oct. 9, 1997) (OSS OII/OIR proceeding). On July 10, 2003, the Commission approved the most recent version of the JPSA in D.03-07-035.

²¹⁶ The "committed order due time" for PM 9 is established by determining the start time of the cut-over and adding the required cut-over interval to that time. The length of time required to complete a particular CHC cut-over depends on the size and complexity of the order, so the interval for coordinated cut-overs tracked in this measure varies.

²¹⁷ The standard for standalone LNP conversions is 98% on time.

²¹⁸ For Measure 9, loop conversions currently are tracked in the submeasure identified as "Coordinated Conversions (excluding LNP)." For Measure 9A, loop conversions are tracked in the following submeasures: "Basic Loops with LNP," "Basic Loops without LNP," "Standalone LNP" and "DSL-capable loops."

²¹⁹ D.02-03-023 and D.02-09-050.

²²⁰ MCI's proposed performance measures were submitted as Attachment 4 to its brief.

²²¹ None of MCI's agreements with SBC affect any of MCI's proposals for the proper standard, interval, level of disaggregation, frequency of data, geographic scope of data (i.e., wire center versus statewide).

²²² MCI Opening Brief 294-309; TRO, ¶ 489; MCI statements at March 22, 2004 performance measure collaborative.