arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.00083stat.MLcs.LG
收藏

分位数回归的灵活模型聚合

Flexible Model Aggregation for Quantile Regression

Amazon Web Services(亚马逊云科技);Seoul National University(首尔国立大学);Cleanlab;Carnegie Mellon University(卡内基梅隆大学)

Rasool Fakoor, Taesup Kim, Jonas Mueller, Alexander J. Smola, Ryan J. Tibshirani

AI总结:

本文研究分位数回归模型的灵活聚合方法,通过允许权重随模型、分位数水平和特征变化,并结合单调性约束与保形校准,在不修改基础模型的前提下提升预测精度和鲁棒性,并在34个数据集上验证了效果。

摘要:

分位数回归是统计学习中的一个基本问题,其动机在于需要对预测中的不确定性进行量化,或者在不进行过度简化的情况下对多样化总体进行建模。例如,流行病学预测、成本估计和收入预测都受益于能够准确量化可能取值的范围。因此,在统计学、机器学习及相关领域多年的研究中,已经为这一问题开发了许多模型。我们并不提出另一种(新的)分位数回归算法,而是采用一种元视角:我们研究聚合任意数量条件分位数模型的方法,以提高准确性和鲁棒性。我们考虑加权集成,其中权重不仅可以在各个模型之间变化,还可以在分位数水平和特征值上变化。本文考虑的所有模型都可以使用现代深度学习工具包进行拟合,因此从实现角度看具有广泛的可访问性和可扩展性。为了提高预测分位数(或等价地,预测区间)的准确性,我们开发了确保分位数保持单调有序的工具,并应用了保形校准方法。这些方法可以在不修改原始基础模型库的情况下使用。我们还回顾了围绕分位数聚合及相关评分规则的一些基本理论,并为该文献贡献了一些新结果(例如,排序后或保序回归后只能改善加权区间分数这一事实)。最后,我们提供了跨两个不同基准库的34个数据集的广泛实证比较。

英文摘要:

Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being overly reductive. For instance, epidemiological forecasts, cost estimates, and revenue predictions all benefit from being able to quantify the range of possible values accurately. As such, many models have been developed for this problem over many years of research in statistics, machine learning, and related fields. Rather than proposing yet another (new) algorithm for quantile regression we adopt a meta viewpoint: we investigate methods for aggregating any number of conditional quantile models, in order to improve accuracy and robustness. We consider weighted ensembles where weights may vary over not only individual models, but also over quantile levels, and feature values. All of the models we consider in this paper can be fit using modern deep learning toolkits, and hence are widely accessible (from an implementation point of view) and scalable. To improve the accuracy of the predicted quantiles (or equivalently, prediction intervals), we develop tools for ensuring that quantiles remain monotonically ordered, and apply conformal calibration methods. These can be used without any modification of the original library of base models. We also review some basic theory surrounding quantile aggregation and related scoring rules, and contribute a few new results to this literature (for example, the fact that post sorting or post isotonic regression can only improve the weighted interval score). Finally, we provide an extensive suite of empirical comparisons across 34 data sets from two different benchmark repositories.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑