IV. Measures of Service Quality

There are several ways the Commission assesses Pacific and Verizon's service quality. Before discussing the carriers' performance on these measures, we lay out here the basic measures of service quality before us in this case.

A. GO 133-B Reporting

1. Introduction

The Commission requires telephone utilities providing service in California to make regular reports pursuant to GO 133-B on performance in the areas of installation, network reliability, trouble reports, installation commitments, and answer times. Telephone utilities, including Pacific and Verizon, are required to submit to the Commission quarterly reports of monthly results on seven direct service performance criteria, as follows:

· two measures related to installation:

o Installation - line energizing commitments met (standard requires that utility meet 95% of commitments).

o Held primary orders over 30 days

· one measure related to network trouble reports:

o network trouble reports per 100 access lines (with a standard requiring fewer than 6.0 reports per 100 lines).

· four measures of carriers' answer times:

o Toll operator answer time (standard requires that each covered traffic office answer 85% of such calls within 10 seconds).

o Directory Assistance operator answer time (standard requires that each covered traffic office answer 85% of such calls within 12 seconds).

o Trouble Report Service Answer Time (TRSAT; standard requires that 80% of such calls be answered within 20 seconds).

o Business Office Answer Time (BOAT; standard requires that 80% of such calls be answered within 20 seconds).⁸

We discuss Pacific's and Verizon's performance on some of these measures below. However, there have been many valid criticisms throughout this proceeding of ambiguities and omissions in GO 133-B. We recently instituted a rulemaking to examine GO 133-B in its entirety as it applies to all carriers.⁹ That rulemaking will consider what changes to existing GO 133-B measures and standards are appropriate. The Commission may use the record of this proceeding to assist it in making its decisions regarding how to revise GO 133-B. However, where it is clear that Pacific or Verizon are not properly interpreting the requirements of GO 133-B, this decision will identify such misinterpretations and order conforming changes.

2. Use of GO 133-B Data to Compare Carriers' Results

The evidence in this proceeding established that data collected under the Order are not comparable among carriers or from year to year. The carriers have made changes from time to time that affect the composition of the data underlying their reported service quality results. Virtually every witness asked agreed that it is difficult to use GO 133-B for performance comparisons across carriers. There are several reasons for this difficulty, as we discuss below.

3. Billing Calls

Prior to February 1999, Pacific included calls related to billing in reporting its BOAT statistics, but ceased doing so thereafter without informing the Commission of the change. As TURN correctly points out, "[t]his change has made it impossible to compare Pacific's BOAT performance (without adjustment) either to its own performance over time, . . . to Verizon's performance . . . or to other carriers' performance . . . ."¹⁰

At the very least, the Commission must be able to rely on GO 133-B data to compare a carrier's performance to itself over time. Currently, however, the carriers' practice does not make such a comparison feasible.

4. DSL-Related Calls

Pacific also once included DSL-related information in its GO 133-B data, but stopped doing so when it moved its DSL functions into a separate subsidiary. The result of this change, once again, means that when we try to compare Pacific's results over time, we end up comparing apples to oranges.

5. Definition of "Primary" Line

There is also disagreement about what GO 133-B means when it requires carriers to report held "primary" service orders. ORA contended the term "primary" means, essentially, that Pacific and Verizon must report data about all basic exchange service lines to a household, regardless of the number of lines at issue. Pacific contended that "primary" refers only to the first line in the house, and not additional lines. However, we also believe GO 133-B should be amended to be more clear. Because a change to GO 133-B would affect other carriers besides Pacific and Verizon, this change would appropriately occur outside this proceeding, and is best addressed in our Service Quality OIR.

6. Automated Response Units

Another general criticism of the GO 133-B reporting is that neither Pacific¹¹ nor Verizon report or track¹² the time a customer spends navigating the companies' Automated Response Units (ARUs) before reaching a live operator. This problem arises both in calculating the carriers' response times in their Business Offices (the BOAT reports) and in connection with their reported Trouble Report Service Answering Time (TRSAT) reporting.

The time a customer spends in "voice mail jail," as some refer to it, may well be as long or longer than the time the customer spends talking to a live operator or service representative. Indeed, since our answer time standards under GO 133-B require "operators," "service attendants" and "business office representatives" to answer calls within mere seconds, it is probable that callers spend more time navigating voice mail menus than during their prescribed seconds-long wait for a company representative.

The evidence substantiated this assumption, at least as to Pacific's residence customers. Pacific stated that the time its residence customers spend in its ARU system ranges from a low of 50 seconds to a high of 300 seconds - that is, from a range of almost 1 minute to 5 minutes.¹³ After that, Pacific places customers in a waiting queue for another 35 seconds on average before reaching a live operator.¹⁴

GO 133-B's failure to address the use of ARUs reflects changes in technology since the Commission adopted the standard, and this technology gap should be closed.¹⁵

7. Held Order Count

GO 133-B requires that carriers report orders that are held - that is, remain pending - for more than 30 days beyond the commitment date ("held orders").¹⁶ Pacific counts such orders once a month, according to its testimony. This creates a result that is inconsistent with GO 133-B's intent that any order older than 30 days be reported to the Commission.

When Pacific's witness Mr. Resnick explained Pacific's practice, it became clear that Pacific does not capture all relevant orders because it counts such orders only once a month. For example, under certain circumstances, Pacific's practice does not count an order that is 48 days overdue as a held order:

Q. Resnick, let's say a customer ordered primary residential service and the commitment date is set for December 29th. We are going to do this as a hypothetical. Due to problems establishing facilities at the customer's residence the line is not installed until February 14th, resulting in a 48-day installation interval from the initial commitment date. Do you have those hypothetical facts in mind, sir?

A. Yes.

Q. In your opinion does this installation meet the GO 133(b) definition of a held order?

A. No.

Q. Why not, sir?

A. . . . [T]he way we measure our GO 133 per the guidelines that are set forth by the Commission, we measure held orders that are held for facilities over 30 days on the 25th of the month. So in this case we would look at January 25th as reporting date for GO 133. We would look back on any orders that were held for more than 30 days past the commitment date. In this case it was not. And so then it would not qualify. The following month, the subsequent month, February 25th, we would look back and this order would have been completed, so therefore it would not count.¹⁷

This method of counting is inconsistent with the requirement of GO 133-B that "An order will count as held when service is not provided within 30 days after commitment date." (Section 3.1(a).) Pacific's method results in it not reporting some orders held up to two months, making its reported performance appear better than its actual performance.

Within 30 days of the effective date of this decision, Pacific shall file a compliance document in this docket indicating that it has conformed its practice to the plain meaning of GO 133-B.¹⁸ Pacific shall, at the very least, change its practice of counting held orders so that it counts such orders as often during the month as is necessary to ensure that all orders for which Pacific does not provide service within 30 days after the commitment date show up in Pacific's held order reporting. It is not acceptable for Pacific to continue its current method of making the count, as that practice causes Pacific to under-report its results.

8. Busy or Abandoned Calls

Finally, GO 133-B does not track busy or abandoned calls. We agree with TURN that a large percentage of either can indicate poor customer service. While some FCC requirements cover these calls, they only do so as part of the time-limited merger monitoring reports we discuss later in this decision. We should address this deficiency in the Service Quality OIR.

B. FCC Automated Reporting Management Information System
(ARMIS) Data

The FCC also requires the carriers to make reports on several aspects of service quality, and the results for relevant years appear in the record of this proceeding.¹⁹ These ARMIS data, as they are called,²⁰ stem from FCC Common Carrier Docket No. 87-313, which implemented service quality reporting requirements for local exchange carriers such as Pacific and Verizon. In 1991, the FCC added specific reports to collect service quality and network infrastructure information.

The ARMIS 43-05 report contains 39 service quality performance measures which track, among other things, whether Pacific or Verizon meet their installation commitments for residential and business customers, trouble reports and repair intervals (e.g., both initial and repeat trouble reports, and the time required to dispatch and complete repairs in response to trouble reports), and switch downtime incidents. While there are no performance standards associated with these reports, they track very important service quality measures.

The ARMIS 43-06 report tracks customer perceptions of Pacific's and Verizon's service quality.²¹ We discuss customer opinion surveys in more detail below.

C. FCC Merger Compliance Oversight Team Data ("MCOT" Data)

Both Pacific and Verizon have undergone changes as a result of large mergers they have entered into with other carriers. As a consequence of these mergers, the FCC has required specific reporting for time-limited periods so that it may monitor service quality impacts that may result from the mergers. (Throughout this proceeding, the parties have referred to these reports generically as "MCOT" requirements, and we use that nomenclature here.)²²

1. MCOT Data - Pacific

As a condition of SBC's merger with Ameritech, the FCC required additional quarterly, state-by-state service quality reporting for the period from June 1999 to November 2002.²³ Categories of reporting for retail services include installation and maintenance, switch outages, transmission facility outages, service quality-related complaints, and answer time performance. The FCC based the reporting categories on the NARUC²⁴ Service Quality White Paper, authored in 1998.²⁵

In late 2000, the FCC notified SBC that, "[t]he quarterly service quality reports filed by SBC Communications, Inc. (`SBC') pursuant to the SBC/Ameritech Merger Order indicate that the quality of service provided by SBC's incumbent local exchange carriers (`LECs') has been deteriorating in several states since approval of the merger in October 1999." The FCC representative went on to state that, "I am concerned that SBC's performance data indicates that consumers in SBC's region are experiencing increasing installation delays, longer repair times, and greater difficulties contacting SBC's incumbent LECs about service quality and other issues. I note also that consumer complaints regarding service quality have increased in recent months in spite of SBC's explicit commitment when the merger was pending to devote greater resources to service quality after the merger closed."²⁶

In its MCOT reporting for the period July 1999 to June 2001, Pacific shows negative spikes in California in the following areas: 1) answer time performance (business customers),²⁷ 2) trouble report rate per 100 lines (especially business customers),²⁸ 3) percentage of installation orders completed within 5 working days (especially residential customers),²⁹ and 4) percentage of installation orders delayed over 30 days (business customers).³⁰

Recognizing the value of the MCOT reporting, during the hearings, Administrative Law Judge (ALJ) Sarah R. Thomas granted TURN's motion seeking an order requiring Pacific to continue to report certain data to this Commission for measures required under the FCC's MCOT requirements that expired in November 2002. (Verizon agreed with TURN voluntarily to continue the reporting until after a final decision in this proceeding.)

Judge Thomas ruled that Pacific should continue to report such information.³¹ She found that Pacific already has a mechanism in place to capture this data easily, that it has no plans to transfer or dismiss the employees who currently prepare the report, and that it would be wasteful to lose the important data the report captures at a time when the Commission is closely examining Pacific's service quality. We hereby ratify that ruling of the judge pursuant to Pub. Util. Code § 310. We require Pacific to continue reporting these results until further notice of the Commission.

2. MCOT Data - Verizon

The FCC also imposed a 36-month reporting requirement as a condition of the 2000 GTE merger with Bell Atlantic that created Verizon.³² As TURN pointed out in a motion filed during Phase 2B, the FCC requirement provides the Commission with information not otherwise available in GO 133-B. For example, while GO 133-B measures the handling of business office calls, it does not track billing calls even though such calls account for half of the calls to the business office.

According to the FCC data,³³ Verizon showed negative spikes or trends in California on several service quality measures at the following times during the period July 2000-June 2001, as compared to the rest of that period: 1) percentage of dissatisfied customers (with business customers reporting 50% dissatisfaction in November 2000 and residential customers reporting 20% dissatisfaction in March 2001),³⁴ 2) answer times (with business answer times in the 50-60 second range in September 2000 and in the 40-50 second range in January 2001 - as compared to a GO 133-B standard of 20 seconds); and residential times exceeding 20 seconds in November 2000 [30 seconds] and January 2001 [40 seconds],³⁵ 3) repair intervals for both residential and business customers spiking in the period January-March 2001,³⁶ 4) repeat trouble reports spiking for both types of customers in March 2001,³⁷ and 5) trouble reports per hundred lines spiking in the same January-March 2001 time period for residential customers.³⁸

While Verizon voluntarily agreed to continue reporting this MCOT data, we will expand on that agreement to make it parallel with Pacific's, and require Verizon to continue to make its MCOT reports to this Commission until further notice.

D. Complaint Data

Customer complaints can also provide a useful indication of carrier service quality performance. There are at least three sources of complaint data in the record of this proceeding: informal complaints filed with the Commission's Consumer Protection and Safety Division, Consumer Affairs Branch (CAB); formal Commission complaints and investigations; and the carriers' own complaint records.

Verizon's own complaint data appear to be more comprehensive than Pacific's. Pacific keeps track of some complaints that come to its Informal Appeals organization but acknowledged that these complaints are but a small fraction of the total complaints it receives. It tends to keep records at the business office level only when the complaint relates to slamming by other carriers and cramming.³⁹ Verizon appears to track all complaints, although a minimal amount of its own data appears in the record of this proceeding.

Both carriers have claimed that we should not take into account formal complaints and investigations, since the Commission will address - or has addressed - the conduct alleged in those proceedings separately. We disagree. The OIR determined that this proceeding would "assess how service quality has fared under NRF." While we do not intend to penalize the carriers twice for the same conduct, this is the first time we have taken a comprehensive look at Pacific and Verizon's service quality since we instituted the NRF regime.

The OIR states that, "requests for relief that are better addressed in complaint proceedings or enforcement OIIs" are beyond the scope of this proceeding, and we do not consider such requests for relief here. However, the scope of this proceeding does not limit the Commission from reviewing the frequency and nature of prior Commission actions addressing carriers' performance and service quality failures during the NRF period as one of many measures used to assess how service quality has fared under NRF.

The records of the other formal Commission proceedings may, examined together, give clues to patterns of behavior that corroborate other service quality results. And the whole may be greater than the sum of its parts if we consider our formal proceedings together rather than individually. The increasing need for Commission intervention to address service quality failures after carriers began operating under NRF may indicate how effective or ineffective NRF is in promoting high quality service.

Any formal proceeding that we have conducted since 1989, when we first stated we would be examining service quality as part of our NRF monitoring process,⁴⁰ is germane to our assessment. Of course, very old cases within that time frame may prove only that at one time Pacific or Verizon had a problem. However, because this assessment covers the entire NRF period, such a case is within the scope of this proceeding.

E. Customer Service Surveys

For this proceeding, ORA conducted a survey of customer perceptions of the service quality of both Pacific and Verizon. We discuss the results below. In addition, the carriers furnished limited information regarding the surveys they conduct themselves, and we also make reference to those surveys in discussing each company's performance. As noted above, the carriers also track customer satisfaction for the FCC as part of their ARMIS 43-06 reporting.

⁸ GO 133-B describes "Business Office Answering Time" (BOAT) as "A measurement of time for the business office representative to answer business office calls." GO 133-B, Section 3.9. ⁹ R.02-12-004, filed Dec. 5, 2002, available at http://www.cpuc.ca.gov/published/final_decision/21982.htm (Service Quality OIR). ¹⁰ TURN Opening/Service Quality at 25. We refer to the parties' briefs in this decision as follows: A party's opening brief is "__'s Opening/Service Quality"; a party's reply brief is "__'s Reply/Service Quality." ¹¹ 23 RT 2973:11-17 (Resnick for Pacific). In this decision, RT refers to the hearing transcripts. Thus, 23 RT 2973:11-17 refers to Volume 23 of the transcript, at page 2973, lines 11-17. ¹² 23 RT 2974:17-23 (Resnick for Pacific; not aware that Pacific can measure how long customers wait in the ARU queue). ¹³ Exh. 2B:139 at 8 n.12 (Piiru Opening Testimony, citing Pacific response to TD data request 02-01-01-1-I (iii). Verizon responded in discovery that it does not track this information "on a regular basis." Id. at 7, n.11. ¹⁴ Id. at 7 & n.12. ¹⁵ Although Pacific asserts it has used ARUs since 1990, it provided no evidence that the Commission was aware of its practice or considered the use of ARUs at the time BOAT and TRSAT measures were adopted in 1992. ¹⁶ GO 133-B, Section 3.1 - Held Primary Service Orders. ¹⁷ 22 RT 2793:24-2794:22 (Resnick). ¹⁸ Parties who believe Pacific has violated GO 133-B may file a complaint based on such a claim and seek relief for any alleged violation. ¹⁹ Exhs. 2B:707 (Verizon) and 2B:704 & 2B:706 (Pacific). ²⁰ The term stands for Automated Reporting Management Information System. ²¹ We discuss customer surveys in the Section entitled "Customer Service Surveys," below. ²² The FCC's Merger Compliance Oversight Team maintains a website reflecting the reported results of Pacific ( http://www.fcc.gov/wcb/mcot/SBC_AIT/) and Verizon ( http://www.fcc.gov/wcb/mcot/BA_GTE/). See also Exhibit (Exh.) 2B:507 at 22-23 (Schilberg Direct Testimony describing MCOT reporting). Exhibit 2B:507 refers to Exhibit 507 from Phase 2B of this proceeding. ²³ FCC 99-279, October 6, 1999, Appendix C, Condition XXIV, ¶ 62, available at http://www.fcc.gov/wcb/mcot/SBC_AIT/compliance_program/. ²⁴ National Association of Regulatory Utility Commissioners. ²⁵ The NARUC Service Quality White Paper is available at http://www.fcc.gov/Bureaus/Common_Carrier/Public_Notices/1999/da992441.txt.

²⁶ Letter from Dorothy Atwood, Chief, FCC Common Carrier Bureau, to Mr. James W. Calloway, Group President - SBC Services, dated October 6, 2000, available at http://www.fcc.gov/wcb/mcot/SBC_AIT/service_quality/. We may take official notice of this letter pursuant to Commission Rule 73.