Monday, March 12, 2012 at 3:12PM Will the FCC Tweak QRA in Response to Economists' Peer Review?
FCC Hints at Including More Robust and Relevant Data in QRA Methodology
A March 9, 2012 Wireline Competition Bureau filing might give hope to the many Quantile Regression Analysis (QRA) critics that the FCC is actually paying attention to the extensive complaints that QRA is fatally flawed. The filing explains that the Wireline Competition Bureau “is currently considering the record received on this topic…In addition, consistent with Office of Management and Budget (OMB) guidelines, the Bureau recently sought peer review of the methodology proposed in Appendix H of the FNPRM.” The Bureau makes available the peer review submissions, and more importantly, a list of additional resources and data sets that it is “considering.”
One of the most pervasive complaints about QRA is that the methodology relies upon data that is inappropriate for depicting the costs of deploying broadband in rural areas. For example, QRA does not account for soil texture of frost index, even though soil texture and frozen ground have tremendous impacts on network deployment and maintenance costs. The FCC initially argued that it would rely on widely available data sets, but was criticized because “widely available” does not necessary mean “accurate or relevant.” In the Bureau’s filing, an extensive list of alternative data is now apparently being considered for the QRA methodology, including:
- USDA SSURGO Soil Survey Data and STATSGO2 Soil Map
- USDA Plant Hardiness Zone Map for climate information
- US Department of Commerce, Census Bureau TigerLine Shape Files for road information
- US Department of the Interior, Fish and Wildlife Service National Wetlands Inventory for climate information
- US Department of the Interior, US Geological Survey Digital Elevation Model Information for topology information and National Hydrography Dataset for water level information
- Study area information filed by various RLECs and PSCs
Is the FCC second-guessing the abhorrent methodology or just following protocol by seeking peer review and considering a more robust data set? A February 21 memorandum from Wireline Competition Bureau chief Sharon Gillett requesting peer review explains that the FCC is “currently considering the best means to implement these regressions”—nearly 4 months after the methodology was adopted in the USF/ICC Transformation Order. The memo continues, “Consistent with OMB peer review guidelines, you should evaluate whether the econometric and economic analyses are reasonable and technically correct, and consistent with accepted practices in the field. You should identify any uncertainties and explain the potential implications of the uncertainties for the technical conclusions drawn, and provide suggestions, if any, for ways to minimize key uncertainties and how the methodology might be improved.” The rural telecom industry, of course, has provided this information and then some in comments—but the FCC is right to seek unbiased analysis from expert economists in addition to the companies that QRA will likely devastate.
A peer review by Office of Strategic Planning and Policy Analysis economist Paroma Sanyal seems generally favorable of using QRA to identify rate-of-return carriers “that may be considered less efficient (higher cost) than their peers, and limiting their payments, to incentivize cost-minimization behavior and efficiency improvements.” However, Sanyal identifies 12 issues with QRA. Some notable critiques include:
- “One major concern with the proposed specification is the underlying assumptions behind the model…applying the quantile regression to the individual cost components may miss some high cost carriers, or mislabel other as high cost.” This phenomenon was explained in reply comments by the Rural Broadband Alliance, which illustrated that some companies invested significantly less than the ninetieth percentile caps in some areas, but were capped nonetheless—in other words, mislabeled as being too high cost.
- “The idea behind capping reimbursements is to incentivize carriers to reduce their costs. However, individual cost capping ignores any complementary or substitutability between the various components…A more flexible approach may be to estimate the ninetieth percentile over the total costs.” This speaks to concerns by RLECs that companies might just shift costs to other categories if they fear being capped in one or more categories—which doesn’t really amount to “efficient investment” by the FCC’s standards.
- Regarding the seemingly-arbitrary decisions on which costs are capped, Sanyal comments “There was little discussion in [Appendix H] why some costs were chosen to be capped, while others were not.”
- Sanyal seems to concur with RLECs that “the specifications my suffer from omitted variable bias, as several important factors that may explain loop costs have not been included in the regression.” Sanyal mentions percentage of bedrock, soil type, roads and streams in the construction area, annual rainfall, and frost-free days. Sanyal comments, “The Nebraska Rural Independent Companies’ Capital Expenditure Study makes a fairly compelling case for including these variables in the regression analysis.”
- Sanyal also questions the ninetieth percentile itself—“It would be interesting to compare the results with regression of other percentiles, and observe whether the effects of the explanatory variables are the same across percentiles.”
Another economist, Tracy Waldon from the Media Bureau, believes that “the method rests on a sound theoretical footing, though it is in need of additional analysis of specific implementation issues;” and “quantile regression is the appropriate tool for the estimation problem at hand.” One interesting issue Waldon brings up is that “in its current form, the Appendix does not make a convincing argument that the existing explanatory variables are sufficient to adequately determine similarly situated study areas. A more convincing presentation would being with a detailed discussion of each of the cost categories and the factors which are likely to drive those costs.” Waldon also provides some analysis about variables, statistical significance vs. economic significance, and mathematical aspects of QRA.
The peer review submissions indicate that RLECs are not just making up the pervasive problems in QRA—apparently even high-level government economists have independently reached similar conclusions. Although these economists generally believe that QRA is an appropriate methodology, as usual, the devil is in the details...However, in this case, RLEC salvation from the current version of QRA may be in the details if the details are widely disputed by economists. Based on the economists' peer review submissions, it is fairly clear that the FCC has a lot of work to do on QRA before it is an acceptable methodology for limiting HCLS.
Do you think the peer review submissions, combined with the overwhelming outcry by RLECs, will be enough to get the FCC to course-correct QRA before significant damage is done to RLEC investment? Will the additional data under consideration at the FCC improve the methodology?





