ALJ/JAR/k47 Mailed 1/22/2001
Decision 01-01-037 January 18, 2001
BEFORE THE PUBLIC UTILITIES COMMISSION OF THE STATE OF CALIFORNIA
Order Instituting Rulemaking on the Commission's Own Motion into Monitoring Performance of Operations Support Systems. |
Rulemaking 97-10-016 (Filed October 9, 1997) |
Order Instituting Investigation on the Commission's Own Motion into Monitoring Performance of Operations Support Systems. |
Investigation 97-10-017 (Filed October 9, 1997) |
INTERIM OPINION ON PERFORMANCE INCENTIVES
INTERIM ORDER ON PERFORMANCE INCENTIVES 1
Summary 2
Background 4
Performance Remedies Plan Fundamentals 6
Initial Proposed Plans 8
Plan Principles 8
Parity and Statistical Model Elements 10
Test for Determining Compliance with Parity 11
Minimum Sample Size 13
Alpha Level/Critical Value 14
Assigned Commissioner's Ruling and Proposed Plan 15
ACR Plan Statistical Model Elements 18
Minimum Sample Size 23
Alpha Level/Critical Value 25
Responses to the ACR Questions and Comments on Its Overall Statistical Model Approach 28
Use of standard Z-test or Modified Z-test 28
Use of Benchmarks without Statistical Tests 29
Use of Special Tables for Benchmark Measures 30
Use of Minimum Sample Size of Thirty 31
Use of Ten Percent Alpha Level versus Fifteen or Five Percent 33
March 2000 Workshop 35
Workshop Recommendations and Positions 37
Hybrid Performance Measurement Plan 37
ORA Performance Measurement Plan 42
Pacific's White Paper Proposal 51
Selection of the Decision Model 55
Decision accuracy 58
Decisions regarding parity measures 59
Determinations regarding benchmarks 68
Statistical models 68
Statistical tests 69
Average-based measures 69
Standard Z-test 70
Modified Z-test 70
Permutation tests 73
Percentage-based measures 79
Rate-based measures 81
Confidence levels 82
Alpha levels 82
Test power 91
Fixed alpha 95
Material differences 97
Optimal alpha and beta levels 100
Minimum sample size 102
Average-based measures 102
Percentage and rate-based measures 111
Data transformations 112
Benchmark issues 115
Benchmark adjustment tables 116
Benchmark statistical testing 121
Benchmark modification 123
Correlation analysis 123
Historical data 124
Identical models for ILECs 124
Payment retroactivity 125
Other issues 127
Z-statistic negative/positive interpretation 127
Interim and permanent models 128
Comments on Draft Decision 130
Findings of Fact 130
Conclusions of Law 138
Appendix A: ACR Questions
Appendix B: References
Appendix C: Decision Model
Appendix D: Fisher's Exact Test
Appendix E: Binomial Exact Test
Appendix F: Beta Error
Appendix G: Balancing Alpha and Beta Error
Appendix H: Pacific's Proposed Aggregation Rules
Appendix I: Implemented Aggregation Rule Results
Appendix J: Log Transformations
Appendix K: Benchmark Small Sample Adjustment Tables
Appendix L: Appearances
