arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.17089stat.MLcs.LG
收藏

高维目标的双参数模型与梯度流的可学习性

Learnability of high-dimensional targets by two-parameter models and gradient flow

Skoltech(斯科尔科沃科技学院)

Dmitry Yarotsky

AI总结:

本文证明当参数少于目标维度时,存在双参数模型可通过梯度流以高概率学习高维目标,但可学习目标集不稠密,且初等函数模型无法实现该可学习性。

摘要:

我们探讨了当模型参数数量W小于目标维度d时,通过梯度流(GF)学习d维目标的理论可能性。我们的主要结果表明,如果目标由特定的d维概率分布描述,那么存在仅含两个参数的模型,能够以任意高的成功概率学习这些目标。另一方面,我们证明当W<d时,必然存在一个大的GF不可学习目标的子集。特别地,可学习目标的集合在R^d中不稠密,且任何与W维球面同胚的R^d子集都包含不可学习的目标。最后,我们观察到,我们主要定理中关于几乎保证双参数学习的模型是通过分层过程构建的,因此不能用单个初等函数表达。我们证明这一限制是本质的,因为大多数用初等函数写成的模型无法实现该定理所展示的可学习性。

英文摘要:

We explore the theoretical possibility of learning $d$-dimensional targets with $W$-parameter models by gradient flow (GF) when $W<d$. Our main result shows that if the targets are described by a particular $d$-dimensional probability distribution, then there exist models with as few as two parameters that can learn the targets with arbitrarily high success probability. On the other hand, we show that for $W<d$ there is necessarily a large subset of GF-non-learnable targets. In particular, the set of learnable targets is not dense in $\mathbb R^d$, and any subset of $\mathbb R^d$ homeomorphic to the $W$-dimensional sphere contains non-learnable targets. Finally, we observe that the model in our main theorem on almost guaranteed two-parameter learning is constructed using a hierarchical procedure and as a result is not expressible by a single elementary function. We show that this limitation is essential in the sense that most models written in terms of elementary functions cannot achieve the learnability demonstrated in this theorem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑