使用广义线性模型学习高斯混合:高维情形下的精确渐近性
Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions
AI总结:
本文研究高维下用任意凸损失与正则化的 ERM 学习一般 $K$ 高斯混合,证明 ERM 估计量的精确渐近性,并用于稀疏均值分类和多类最大间隔相变分析。
摘要:
用于多类分类问题的广义线性模型是现代机器学习任务的基本构建模块之一。在本文中,我们通过采用任意凸损失和正则化的经验风险最小化(ERM),刻画了对具有一般均值和协方差的 $K$ 个高斯混合的学习。特别地,我们证明了在高维情形下刻画 ERM 估计量的精确渐近性,扩展了文献中若干此前关于高斯混合分类的结果。我们在统计学习中两个受关注的任务上例证了我们的结果:a)稀疏均值混合的分类,其中我们研究了 $\ell_1$ 惩罚相对于 $\ell_2$ 的效率;b)最大间隔多类分类,其中我们刻画了当 $K>2$ 时多类 logistic 最大似然估计量存在性的相变。最后,我们讨论了我们的理论如何能够应用于合成数据范围之外,表明在不同情形下,高斯混合能够紧密捕捉真实数据集中分类任务的学习曲线。
英文摘要:
Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of $K$ Gaussians with generic means and covariances via empirical risk minimisation (ERM) with any convex loss and regularisation. In particular, we prove exact asymptotics characterising the ERM estimator in high-dimensions, extending several previous results about Gaussian mixture classification in the literature. We exemplify our result in two tasks of interest in statistical learning: a) classification for a mixture with sparse means, where we study the efficiency of $\ell_1$ penalty with respect to $\ell_2$; b) max-margin multi-class classification, where we characterise the phase transition on the existence of the multi-class logistic maximum likelihood estimator for $K>2$. Finally, we discuss how our theory can be applied beyond the scope of synthetic data, showing that in different cases Gaussian mixtures capture closely the learning curve of classification tasks in real data sets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。