发表机构
College of Sciences, Northeastern University; School of Mathematics and Statistics, Xi’an Jiao Tong University（东北大学理学院; 西安交通大学数学与统计学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对高维无岭最小二乘插值，基于广义尖峰协方差模型，揭示了其预测行为由回归系数与尖峰特征空间的对齐程度决定，刻画了尖峰特性对双下降现象的影响，为过参数化回归的泛化提供了统一认识。
AI 中文摘要
本文研究了当特征维度$p$与样本量$n$成比例增长时，高维无岭最小二乘估计量的样本外预测风险的渐近行为。我们考虑具有多个潜在因子的广义尖峰总体协方差模型，其中尖峰特征值的数量可保持有限或随$n$增长，且尖峰特征值可有界或以任意速率发散。除了刻画协方差谱的影响外，我们揭示了良性过拟合背后的新机制：无岭插值的预测行为根本上由回归系数$\boldsymbol\beta$与总体协方差矩阵的尖峰特征空间之间的对齐程度决定。特别地，我们表明沿潜在尖峰方向分布的信号能量决定了插值会导致良性、适度还是灾难性的过拟合。我们的理论框架在最小矩条件下建立了精确的预测风险极限，仅要求有限的四阶矩而非高斯性。我们刻画了尖峰的数量、强度和几何结构如何共同影响双下降现象。这些结果为理解过参数化回归中潜在协方差结构何时促进或阻碍泛化提供了统一的认识。
英文摘要
This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension $p$ and the sample size $n$ grow proportionally. We consider a generalized spiked population covariance model with multiple latent factors, where the number of spiked eigenvalues may remain finite or increase with $n$, and the spiked eigenvalues may be bounded or diverge at arbitrary rates. Beyond characterizing the impact of covariance spectra, we reveal a new mechanism underlying benign overfitting: the prediction behavior of ridgeless interpolation is fundamentally governed by the alignment between the regression coefficient $\boldsymbolβ$ and the spiked eigenspaces of the population covariance matrix. In particular, we show that the signal energy distributed along latent spike directions determines whether interpolation leads to benign, tempered, or catastrophic overfitting. Our theoretical framework establishes sharp prediction risk limits under minimal moment conditions, requiring only finite fourth moments rather than Gaussianity. We characterize how the number, strength, and geometric structure of the spikes jointly influence the double-descent phenomenon. These results provide a unified understanding of when latent covariance structures facilitate or hinder generalization in overparameterized regression.
