逐项独立同分布重尾数据上经验风险最小化的渐近分析

中文摘要 AI 辅助

许多现实世界的数据集出现异常大值的频率远高于高斯模型所预测的。重尾分布捕捉了这种行为，但在重尾分布下评估学习性能仍然具有挑战性，因为罕见的大特征项即使在高维情况下也保留着不可忽略的影响。即使在逐项独立同分布对称α稳定数据的线性回归经验风险最小化的典型设置中，预测的精确渐近特征也一直缺乏。在这项工作中，我们引入了一个函数序参量，它描述了与每个系数相关的随机有效问题。利用复制方法，我们在比例高维极限下完全刻画了泛化误差，其中样本量和特征维度以固定比率发散。此外，该分析建立了重尾普适性定律、将典型误差与预测可靠性联系起来的标度律，以及贝叶斯最优预测误差。除了刻画极端项对学习过程的影响外，我们的方法还广泛适用于其他具有持久局部异质性的系统。

英文摘要

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric $α$-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

发表机构

• Graduate School of Science, The University of Tokyo（东京大学理学研究生院）
• Institute for Physics of Intelligence, The University of Tokyo（东京大学智能物理研究所）
• Trans-Scale Quantum Science Institute, The University of Tokyo（东京大学跨尺度量子科学研究所）

机构由 AI 辅助整理，请以论文原文为准。