arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.07774cs.LGcs.CCcs.DSmath.AGmath.STstat.TH
收藏

多项式近零集的小覆盖与潜变量模型学习

Small Covers for Near-Zero Sets of Polynomials and Learning Latent Variable Models

UW Madison(威斯康星大学麦迪逊分校);University of California, San Diego(加利福尼亚大学圣迭戈分校)

Ilias Diakonikolas, Daniel M. Kane

摘要:

设$V$是任意多元次数的向量空间- $d$齐次多项式,余维数至多为$k$,$S$是$V$ { \em}中所有多项式几乎为零的点集。在$\ell_2$ -范数下,我们建立了$S$的$ε$ -覆盖的大小的定性最优上界.粗略地说,我们证明了基数为$M = (k/ε)^{O_d(k^{1/d})}$的$S$存在一个$ε$ -覆盖.我们的结果是建设性的产生一个算法来计算这样一个$ε$ -覆盖,运行在时间$\mathrm{poly}(M)$。在我们的结构结果的基础上,我们获得了显着改进的学习算法的几个基本的高维概率模型的隐变量。这些包括$k$的密度和参数估计-球形高斯混合(具有已知的公共协方差),PAC学习具有$k$隐藏单元(高斯分布下)的单隐藏层ReLU网络,$k$的密度和参数估计-线性回归的混合(具有高斯协变量),以及$k$的参数估计-超平面的混合。我们的算法运行在时间{ \em准多项式}中的参数$k$。这些问题的以前的算法在$k^{Ω(1)}$中具有指数的运行时间。在高层次上,我们的算法对所有这些学习问题的工作原理如下:通过计算隐藏参数的低阶矩,我们能够找到一个向量空间的多项式,几乎消失的未知参数。我们的结构结果使我们能够计算一个准多项式大小的覆盖隐藏参数,我们利用我们的学习算法的集合。

英文摘要:

Let $V$ be any vector space of multivariate degree-$d$ homogeneous polynomials with co-dimension at most $k$, and $S$ be the set of points where all polynomials in $V$ {\em nearly} vanish. We establish a qualitatively optimal upper bound on the size of $ε$-covers for $S$, in the $\ell_2$-norm. Roughly speaking, we show that there exists an $ε$-cover for $S$ of cardinality $M = (k/ε)^{O_d(k^{1/d})}$. Our result is constructive yielding an algorithm to compute such an $ε$-cover that runs in time $\mathrm{poly}(M)$. Building on our structural result, we obtain significantly improved learning algorithms for several fundamental high-dimensional probabilistic models with hidden variables. These include density and parameter estimation for $k$-mixtures of spherical Gaussians (with known common covariance), PAC learning one-hidden-layer ReLU networks with $k$ hidden units (under the Gaussian distribution), density and parameter estimation for $k$-mixtures of linear regressions (with Gaussian covariates), and parameter estimation for $k$-mixtures of hyperplanes. Our algorithms run in time {\em quasi-polynomial} in the parameter $k$. Previous algorithms for these problems had running times exponential in $k^{Ω(1)}$. At a high-level our algorithms for all these learning problems work as follows: By computing the low-degree moments of the hidden parameters, we are able to find a vector space of polynomials that nearly vanish on the unknown parameters. Our structural result allows us to compute a quasi-polynomial sized cover for the set of hidden parameters, which we exploit in our learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑