arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.01583stat.MLcs.ITcs.LGmath.ITmath.PRmath.STstat.TH
收藏

多指标模型的谱估计量:精确渐近性与最优弱恢复

Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery

Institute of Science and Technology Austria(奥地利科学与技术研究院)

Filip Kovačević, Yihan Zhang, Marco Mondelli

AI总结:

本文研究多指标模型中的信号子空间恢复问题,通过精确刻画谱矩阵前p个特征值与特征向量重叠的渐近行为及相变,确定实现最优弱恢复、样本需求最小的谱估计量。

摘要:

多指标模型为研究具有低维结构的函数的可学习性提供了一个常用框架,并且由于其与神经网络的联系,近来已成为密集研究的对象。在本文中,我们专注于通过谱估计量恢复由信号张成的子空间;谱估计量是实践中常用的一类方法,常作为迭代算法的热启动。我们的主要技术贡献是:当样本量和输入维度成比例增长、而待恢复空间的维度 p 固定时,对谱方法的性能给出精确的渐近刻画。具体而言,我们定位了谱矩阵的前 p 个特征值,并建立了相应特征向量(即谱估计量)与信号子空间的一组基之间的重叠。我们的分析揭示了一种相变现象:随着样本复杂度增加,特征值会脱离谱的主体,而当这种情况发生时,特征向量会恢复出所需子空间的方向。我们提出的精确刻画使数据预处理的优化成为可能,从而能够识别出弱恢复所需样本量最小的谱估计量。

英文摘要:

Multi-index models provide a popular framework to investigate the learnability of functions with low-dimensional structure and, also due to their connections with neural networks, they have been object of recent intensive study. In this paper, we focus on recovering the subspace spanned by the signals via spectral estimators -- a family of methods routinely used in practice, often as a warm-start for iterative algorithms. Our main technical contribution is a precise asymptotic characterization of the performance of spectral methods, when sample size and input dimension grow proportionally and the dimension $p$ of the space to recover is fixed. Specifically, we locate the top-$p$ eigenvalues of the spectral matrix and establish the overlaps between the corresponding eigenvectors (which give the spectral estimators) and a basis of the signal subspace. Our analysis unveils a phase transition phenomenon in which, as the sample complexity grows, eigenvalues escape from the bulk of the spectrum and, when that happens, eigenvectors recover directions of the desired subspace. The precise characterization we put forward enables the optimization of the data preprocessing, thus allowing to identify the spectral estimator that requires the minimal sample size for weak recovery.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑