发表机构
University College London; Los Alamos National Laboratory; King’s College London（伦敦大学学院; 洛斯阿拉莫斯国家实验室; 伦敦国王学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对含Ridge、Lasso等正则化器的线性回归，采用零温复制方法解析计算最小训练损失的大偏差统计量，数值模拟验证其与典型波动高斯尾部存在显著偏差。
AI 中文摘要
线性回归是从数据中学习模式的最简单且应用最广泛的工具之一：它拟合一组系数，使得预测变量的线性组合能最佳匹配观测响应。拟合质量由残差平方和衡量，即预测值与数据之间的总平方误差，其最小值定义了训练损失。我们考虑高斯设计与噪声，教师系数从一般分布$p(\eta)$中独立抽取，以及一类可分正则化器，包括Ridge（岭回归）和Lasso（套索回归）。利用零温复制方法，我们针对预测变量数量为$P$、观测数量为$N$且$r=P/N$固定的情况，解析计算了最小训练损失的大偏差统计量。我们得到的速率函数描述了最优损失在不同样本间的罕见波动。大量数值模拟与我们的理论吻合极好，且清晰显示出其与典型波动高斯 regime（ regime 译为“ regime”，保留专业术语）尾部的显著偏差。
英文摘要
Linear regression is one of the simplest and most widely used tools to learn patterns from data: it fits a set of coefficients so that a linear combination of predictors best matches observed responses. The quality of the fit is measured by the residual sum of squares, the total squared mismatch between predictions and data, whose minimum defines the training loss. We consider Gaussian design and noise, with teacher coefficients independently drawn from a general distribution $p(β)$, and a general class of separable regularizers, including Ridge and Lasso. Using the zero-temperature replica method, we compute analytically the large-deviation statistics of the minimum training loss for large numbers $P$ of predictors and $N$ of observations, with $r=P/N$ fixed. The rate function we compute governs rare sample-to-sample fluctuations of the optimal loss. Extensive numerical simulations are in excellent agreement with our theory and clearly show a pronounced deviation from the Gaussian regime of typical fluctuations in the tails.
Comments7 pages, 2 figs in main text. Supplemental material included
