发表机构
Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究仿射估计量的凸聚合与线性聚合问题，通过最小化Mallows' C_p准则估计权重，建立了达到最优的极小极大聚合速率。
AI 中文摘要
估计过程的聚合在计量经济学、统计学和机器学习中有着重要的应用。经典统计聚合理论主要关注一种“纯聚合”设定，其中候选估计量要么是确定性的，要么是使用与聚合所用样本独立的留出样本构建的。当候选估计量和聚合权重从同一数据集估计而不进行样本分割时，Bellec [\textit{Ann. Statist.} \textbf{46} (2018), 30--59] 建立了仿射估计量模型选择聚合的最优速率。然而，在该设定之外，关于最优聚合速率以及达到这些速率的聚合规则的构造等基本问题在很大程度上仍未解决。在本文中，我们考虑聚合有限个仿射估计量以学习它们的最优凸组合的问题。该框架涵盖了统计学和机器学习中广泛使用的一类丰富的估计量，包括最小二乘估计量、核岭估计量、随机特征回归估计量等。我们建立了仿射估计量凸聚合的极小极大速率。特别地，我们表明通过最小化 Mallows' $C_p$ 准则来估计权重可以达到最优速率。我们进一步研究了具有无约束权重的线性聚合设定，并在适当的仿射估计量类别上建立了匹配的极小极大下界和上界。
英文摘要
Aggregation of estimation procedures has found important applications in econometrics, statistics, and machine learning. Classical statistical aggregation theory has mainly focused on a \emph{pure aggregation} setting, where the candidate estimators are either deterministic or constructed using a held-out sample independent of that used for aggregation. When the candidate estimators and the aggregation weights are estimated from the same dataset without sample splitting, Bellec [\emph{Ann. Statist.} \textbf{46} (2018), 30--59] established the optimal rate for model-selection aggregation of affine estimators. Beyond this regime, however, fundamental questions regarding the optimal aggregation rates and the construction of aggregation rules attaining these rates remain largely unresolved. In this paper, we consider the problem of aggregating a finite collection of affine estimators to learn an optimal convex combination of them. This framework encompasses a rich class of estimators widely used in statistics and machine learning, including least squares estimators, kernel ridge estimators, random feature regression estimators, and among many others. We establish the minimax rate for convex aggregation of affine estimators. In particular, we show that estimating the weights by minimizing a Mallows' $C_p$ criterion attains the optimal rate. We further study the linear aggregation regime with unrestricted weights and establish matching minimax lower and upper bounds over suitable classes of affine estimators.
