I. Parity measures
A. Statistical Tests
All statistical tests will be one-tailed tests.
1. Average-based Parity Measures
The Modified t-test will be used for all average-based parity measures as specified in:
Brownie, C., Boos, D., & Hughes-Oliver, J. (1990). Modifying the t and ANOVA F tests when treatment is expected to increase variability relative to controls. Biometrics, 46, 259-266.
The Modified t-test for the difference in means (averages) between the ILEC and the CLEC populations is:
t = (Mi-Mc)/[Si*sqrt(1/Nc+1/Ni)]
Where:
Mc = the CLEC mean result
Mi = the ILEC mean result
Si = the standard deviation of the results for the ILEC
Nc = the CLEC sample size
Ni = the ILEC sample size
sqrt = square root
For measures of time intervals, the raw score distribution will be normalized by taking the natural log of each score after a constant of 0.4 of the smallest unit of measurement is added to each score. For example, if the smallest unit of measurement is an integer, then the added constant would be 0.4:
xtran = ln(x + 0.4)
Similarly, if the smallest unit of measurement is 0.01, then the added constant would be 0.004:
xtran = ln(x + 0.004)
Results that are not measures of time intervals (e.g., Measure 34) will not be transformed. Results for Measure 44 will not be transformed.
The Modified t-test calculation for average parity measures will be structured so that a negative sign indicates "worst" performance. Specifically, when a lower value represents better performance, such as time to provision a service, the CLEC mean will be subtracted from the ILEC mean. Different performance measures may require reversing the means in the equation to have a negative sign indicate poorer performance.
The t-statistic will be converted to a p-value (probability value) using a t-distribution table or calculation. Degrees of freedom (df) will be based only on the ILEC sample size consistent with Brownie, et al. If the obtained p-value is less than the critical alpha (__ value, then the result will be deemed not in parity.
2. Proportion Parity Measures
The Fisher's Exact Test will be used for all percentage or proportion parity measures as specified in:
Sheskin, D. (1997). Handbook of parametric and nonparametric statistical procedures. Boca Raton: CRC Press, pp. 221-225.
If the obtained p-value is less than the critical _ value, then the result will be deemed out-of-parity.
3. Rate-based Parity Measures
The Binomial Exact Test will be used for all rate parity measures. The Binomial Exact Test is specified in GTECs Exhibit C, Section 3, "Permutation Test for Rates", Equations 3.1 and 3.2 (Deliverable #7, Facilitated Work Group, April 2000).
4. Indexed-based Parity Measures
Measure 42 provides an index of parity performance that will be assessed by comparing ILEC and CLEC performance as follows:
Non-parity will be identified when the ILEC percentage minus the CLEC percentage exceeds 0.05 percentage points.
B. Critical Alpha Level for Parity Tests
The p-values obtained from the parity statistical tests will be compared to the critical alpha values as specified below. A performance result with a p-value less than the critical alpha will be deemed a performance failure. The critical alphas to be applied are listed below:
For Tier I:
Examine the single-month industry aggregate using:
· 0.10 for sample sizes of 1 to 499.
· 0.05 for sample sizes of 500 and greater.
For CLEC-level analyses:
For multiple-month tests:
· Use 0.20 for the test for each and every individual month (i.e., Chronic: months 1, 2, and 3. Extended: months 1, 2, 3, 4, 5, and 6).
For single-month tests:
If the industry aggregate fails:
· For each CLEC with a sample size of 1 to 29 use 0.20.
· For each CLEC with a sample size of 30 to 499 use 0.10.
· For each CLEC with a sample size of 500 or greater, use 0.05.
If the industry aggregate passes:
· For each CLEC with a sample size of 1 to 99 use 0.10.
· For each CLEC with a sample size of 100 or greater, use 0.05.
For Tier II:
Since all Tier II tests are repeated failure tests, use 0.20 for the test for each and every individual month (i.e., months 1, 2, and 3). (Note: the single-month aggregate failure rate used as a multiplier for calculating the payment amounts will follow the single-month industry aggregate test rules listed above.)
C. Sample Sizes and Aggregation Rules
Statistical tests will be applied to the monthly performance results specified in the Joint Partial Settlement Agreement (D.01-05-087 or "JPSA") and in any Commission-approve modifications to the JPSA. Statistical analyses and decision rules will be applied to determine performance subject to the performance incentives plan for all samples regardless of sample size.
D. Measures without Retail Analogues.
In months where there are no retail analogue performance data, the prior six months of ILEC data be aggregated (to the extent that such data exist) and used in place of the data-deficient month. If the aggregate does not produce sufficient ILEC data, the sub-measure will not be evaluated for the month.
II. Benchmark Measures
For large samples, the actual performance will be compared to the benchmark nominal percentage according to the percentage set in the Joint Partial Settlement Agreement approved by the Commission. For small samples, maximum permitted "misses" shall be determined by small sample adjustment tables. Small samples are defined as follows:
90 percent benchmarks - 50 cases or less
95 percent benchmarks - 100 cases or less
98 percent benchmarks - 250 cases or less
99 percent benchmarks - 500 cases or less
99.65 (and 0.0035) percent benchmarks - 1429 cases or less
99.75 (and 0.0025) percent benchmarks - 2000 cases or less
SMALL SAMPLE ADJUSTMENT TABLES | ||||||||||||
|
Benchmark = 90% |
Benchmark = 95% |
Benchmark = 98% |
Benchmark = 99% |
Benchmark = 99.65% |
Benchmark = 99.75% | ||||||
Maximum Permitted Misses |
Minimum Sample Size |
Maximum Sample Size |
Minimum Sample Size |
Maximum Sample Size |
Minimum Sample Size |
Maximum Sample Size |
Minimum Sample Size |
Maximum Sample Size |
Minimum Sample Size |
Maximum Sample Size |
Minimum Sample Size |
Maximum Sample Size |
0 |
1 |
1 |
1 |
3 |
1 |
9 |
1 |
19 |
1 |
55 |
1 |
77 |
1 |
2 |
9 |
4 |
19 |
10 |
48 |
20 |
97 |
56 |
304 |
78 |
390 |
2 |
10 |
20 |
20 |
40 |
49 |
101 |
98 |
202 |
305 |
631 |
391 |
808 |
3 |
21 |
31 |
41 |
63 |
102 |
159 |
203 |
319 |
632 |
999 |
809 |
1279 |
4 |
32 |
44 |
64 |
88 |
160 |
222 |
320 |
445 |
1000 |
1393 |
1280 |
1783 |
5 |
45 |
50 |
89 |
100 |
223 |
250 |
446 |
500 |
1394 |
1429 |
1784 |
2000 |
The small sample adjustment tables shall be used in the following steps:
1. The number of performance "misses" for the CLEC industry-wide aggregate for each remedy plan benchmark sub-measure will be compared to the number of permitted misses for all sample sizes covered by the related adjustment table. Industry aggregate performance will be identified as passing if the number of actual misses is less than or equal to the number of permitted misses, and identified as failing if otherwise.
2. For CLEC industry-wide aggregate sample sizes not covered by the related adjustment table, the actual performance percentage result will be compared to the benchmark nominal percentage value. Industry aggregate performance will be identified as passing if the actual performance percentage result is greater than or equal to the benchmark nominal percentage value, and identified as failing if otherwise.
3. For each sub-measure where the CLEC industry-wide aggregate performance fails the benchmark, the actual performance percentage result for each non-aggregated CLEC result will be compared to the benchmark nominal percentage value. Each individual performance result will be identified as passing if the actual performance percentage result is greater than or equal to the benchmark nominal percentage value, and identified as failing if otherwise.
4. For sample sizes covered by the related adjustment table where the CLEC industry-wide aggregate performance passes the benchmark, the following shall apply for each sub-measure. For each benchmark sub-measure, the number of performance "misses" for each non-aggregated CLEC will be compared to the number of permitted misses. CLEC performance will be identified as passing if the number of actual misses is less than or equal to the number of permitted misses, and identified as failing if otherwise.
5. For sample sizes not covered by the related adjustment table where the CLEC industry-wide aggregate performance passes the benchmark, the following shall apply. The actual performance percentage result for each non-aggregated CLEC result will be compared to the benchmark nominal percentage value. Each individual performance result will be identified as passing if the actual performance percentage result is greater than or equal to the benchmark nominal percentage value, and identified as failing if otherwise.
Small Sample Adjustment Table
Calculation Procedure
1. Set the benchmark to B. In this procedure it is assumed that B is a number close to 1.0. If the benchmark is small, simply use 1 - B.
2. Set the maximum length of the table, L, according to the formula
3. Set the derivation (reference) sample size according the formula
4. Calculate the implied performance level, P, as that value which solves the equation
5. Calculate the permitted number of misses, m for the sample size n, as the largest value of k that satisfies the following:
