高斯数据是你所需要的吗?高维广义线性估计普适性的程度和极限
Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation
摘要:
本文研究了单指标模型下混合高斯数据的广义线性估计问题。我们的第一个结果是一个尖锐的渐近表达式的测试和训练误差在高维制度。受最近关于广义线性估计中的高斯检验和训练误差的普遍性的结果流的启发,我们问自己一个问题:“什么时候一个单一的高斯足以描述误差?".我们的公式使我们能够在积极和消极的方向上对这个问题给出尖锐的答案。更确切地说,我们表明,高斯普适性(或缺乏)的充分条件关键取决于目标权重和混合聚类的均值和协方差之间的对齐,我们精确地量化。在最小二乘插值的特殊情况下,我们证明了训练误差的强普适性,并表明它遵循一个简单的,封闭形式的表达式。最后,我们将我们的结果应用到真实的数据集,澄清了最近在文献中关于高斯普遍性的错误在这种情况下的一些讨论。
英文摘要:
In this manuscript we consider the problem of generalized linear estimation on Gaussian mixture data with labels given by a single-index model. Our first result is a sharp asymptotic expression for the test and training errors in the high-dimensional regime. Motivated by the recent stream of results on the Gaussian universality of the test and training errors in generalized linear estimation, we ask ourselves the question: "when is a single Gaussian enough to characterize the error?". Our formula allow us to give sharp answers to this question, both in the positive and negative directions. More precisely, we show that the sufficient conditions for Gaussian universality (or lack of thereof) crucially depend on the alignment between the target weights and the means and covariances of the mixture clusters, which we precisely quantify. In the particular case of least-squares interpolation, we prove a strong universality property of the training error, and show it follows a simple, closed-form expression. Finally, we apply our results to real datasets, clarifying some recent discussion in the literature about Gaussian universality of the errors in this context.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。