arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.04360cs.LGmath.STstat.MLstat.TH
收藏

$L^p$范数的一般逼近下界及其在前馈神经网络中的应用

A general approximation lower bound in $L^p$ norm, with applications to feed-forward neural networks

Institut de Mathématiques de Toulouse(图卢兹数学研究所);Université de Toulouse(图卢兹大学);CNRS(法国国家科学研究中心);IRT Saint Exupéry(圣埃克苏佩里技术研究所)

El Mehdi Achour, Armand Foucault, Sébastien Gerchinovitz, François Malgouyres

摘要:

我们研究了神经网络表达能力的基本限制。给定两个实值函数集合$F$,$G$,我们首先证明了对于任意$p \geq 1$和任意概率测度$μ$,$F$中的函数在$L^p(μ)$范数下如何被$G$中的函数逼近的一般下界.下限取决于$F$的包装数、$F$的范围和$G$的脂肪粉碎维度。然后,我们实例化的情况下,$G$对应于一个分段多项式前馈神经网络的约束,并详细描述了应用程序的两个集合$F$:H{<$}lder球和多元单调函数。除了将(已知或新的)上限匹配到对数因子之外,我们的下限还揭示了$L^p$范数或sup范数近似之间的相似性或差异,解决了DeVore等人的一个悬而未决的问题。(2021)。我们的证明策略不同于sup范数的情况,并使用了Mendelson(2002)的关键概率结果。

英文摘要:

We study the fundamental limits to the expressive power of neural networks. Given two sets $F$, $G$ of real-valued functions, we first prove a general lower bound on how well functions in $F$ can be approximated in $L^p(μ)$ norm by functions in $G$, for any $p \geq 1$ and any probability measure $μ$. The lower bound depends on the packing number of $F$, the range of $F$, and the fat-shattering dimension of $G$. We then instantiate this bound to the case where $G$ corresponds to a piecewise-polynomial feed-forward neural network, and describe in details the application to two sets $F$: H{ö}lder balls and multivariate monotonic functions. Beside matching (known or new) upper bounds up to log factors, our lower bounds shed some light on the similarities or differences between approximation in $L^p$ norm or in sup norm, solving an open question by DeVore et al. (2021). Our proof strategy differs from the sup norm case and uses a key probability result of Mendelson (2002).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑