ALJ/JAR/k47 DRAFT H-7
1/18/2001
Decision
BEFORE THE PUBLIC UTILITIES COMMISSION OF THE STATE OF CALIFORNIA
Order Instituting Rulemaking on the Commission's Own Motion into Monitoring Performance of Operations Support Systems. |
Rulemaking 97-10-016 (Filed October 9, 1997) |
Order Instituting Investigation on the Commission's Own Motion into Monitoring Performance of Operations Support Systems. |
Investigation 97-10-017 (Filed October 9, 1997) |
INTERIM OPINION ON PERFORMANCE INCENTIVES
INTERIM ORDER ON PERFORMANCE INCENTIVES 1
Summary 2
Background 3
Performance Remedies Plan Fundamentals 6
Initial Proposed Plans 8
Plan Principles 8
Parity and Statistical Model Elements 9
Test for Determining Compliance with Parity 11
Minimum Sample Size 12
Alpha Level/Critical Value 13
Assigned Commissioner's Ruling and Proposed Plan 15
ACR Plan Statistical Model Elements 17
Minimum Sample Size 22
Alpha Level/Critical Value 24
Responses to the ACR Questions and Comments on Its Overall Statistical Model Approach 27
Use of standard Z-test or Modified Z-test 27
Use of Benchmarks without Statistical Tests 28
Use of Special Tables for Benchmark Measures 29
Use of Minimum Sample Size of Thirty 30
Use of Ten Percent Alpha Level versus Fifteen or Five Percent 32
March 2000 Workshop 34
Workshop Recommendations and Positions 36
Hybrid Performance Measurement Plan 36
ORA Performance Measurement Plan 41
Pacific's White Paper Proposal 49
Selection of the Statistical Model 53
Decision accuracy 57
Decisions regarding parity measures 58
Determinations regarding benchmarks 67
Statistical models 67
Statistical tests 68
Average-based measures 68
Standard Z-test 69
Modified Z-test 69
Permutation tests 71
Percentage-based measures 77
Rate-based measures 79
Confidence levels 80
Alpha levels 80
Test power 88
Fixed alpha 91
Material differences 93
Optimal alpha and beta levels 95
Minimum sample size 97
Average-based measures 97
Percentage and rate-based measures 106
Data transformations 107
Benchmark issues 110
Benchmark adjustment tables 111
Benchmark statistical testing 115
Benchmark modification 117
Correlation analysis 117
Historical data 118
Identical models for ILECs 118
Payment retroactivity 119
Other issues 120
Z-statistic negative/positive interpretation 120
Interim and permanent models 122
Comments on Draft Decision 122
Findings of Fact 123
Conclusions of Law 130
Appendix A: ACR Questions
Appendix B: References
Appendix C: Decision Model
Appendix D: Fisher's Exact Test
Appendix E: Binomial Exact Test
Appendix F: Beta Error
Appendix G: Balancing Alpha and Beta Error
Appendix H: Pacific's Proposed Aggregation Rules
Appendix I: Implemented Aggregation Rule Results
Appendix J: Log Transformations
Appendix K: Benchmark Small Sample Adjustment Tables
Appendix L: Appearances
