arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.16003cs.LGcs.CCcs.DSstat.ML
收藏

学习(非常)简单的生成模型很难

Learning (Very) Simple Generative Models Is Hard

UC Berkeley(加州大学伯克利分校);Microsoft Research(微软研究院);CMU(卡内基梅隆大学)

Sitan Chen, Jerry Li, Yuanzhi Li

摘要:

受最近深度生成模型的经验成功的启发,我们研究了以下无监督学习问题的计算复杂性。对于一个未知的神经网络$F:\mathbb{R}^d\to\mathbb{R}^{d'}$,令$D$是通过将标准高斯$\mathcal{N}(0,\textrm{Id}_d)$推过$F$给出的$\mathbb{R}^{d'}$上的分布。注射了。样本来自$D$,目标是输出统计距离接近$D$的任何分布。我们在统计查询(SQ)模型下证明,即使$F$的输出坐标是具有$\log(d)$神经元的一个隐藏层ReLU网络,也没有多项式时间算法可以解决这个问题。以前,这个问题的最佳下限只是遵循监督学习的下限,并且需要至少两个隐藏层和$\mathrm{poly}(d)$神经元[Daniely-Vardi '21,Chen-Gollakota-Klivans-Meka '22]。在我们的证明中的关键成分是一个基于ODE的构造一个紧支持的,分段线性函数$f$与多项式有界的斜率,使下推的$\mathcal{N}(0,1)$ $f$匹配所有低阶矩的$\mathcal{N}(0,1)$。

英文摘要:

Motivated by the recent empirical successes of deep generative models, we study the computational complexity of the following unsupervised learning problem. For an unknown neural network $F:\mathbb{R}^d\to\mathbb{R}^{d'}$, let $D$ be the distribution over $\mathbb{R}^{d'}$ given by pushing the standard Gaussian $\mathcal{N}(0,\textrm{Id}_d)$ through $F$. Given i.i.d. samples from $D$, the goal is to output any distribution close to $D$ in statistical distance. We show under the statistical query (SQ) model that no polynomial-time algorithm can solve this problem even when the output coordinates of $F$ are one-hidden-layer ReLU networks with $\log(d)$ neurons. Previously, the best lower bounds for this problem simply followed from lower bounds for supervised learning and required at least two hidden layers and $\mathrm{poly}(d)$ neurons [Daniely-Vardi '21, Chen-Gollakota-Klivans-Meka '22]. The key ingredient in our proof is an ODE-based construction of a compactly supported, piecewise-linear function $f$ with polynomially-bounded slopes such that the pushforward of $\mathcal{N}(0,1)$ under $f$ matches all low-degree moments of $\mathcal{N}(0,1)$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑