面向浅层神经网络的广义Barron空间
On Generalized Barron Spaces for Shallow Neural Networks
AI总结：
本研究针对通用激活函数的浅层神经网络，提出广义Barron空间$B_{\sigma}^{\varphi}$，证明其可嵌入Sobolev空间、涵盖多数传统空间，推导逼近速率与数值微分误差界，数值例子验证其可构建不同光滑度的同激活函数神经网络。
AI中文摘要：
经典Barron空间是专门为主要使用ReLU、$\mathrm{ReLU}^k$（RePU）或Lipschitz连续激活函数的浅层神经网络设计的函数空间。本研究针对具有特定光滑性性质的通用激活函数的浅层神经网络，引入广义Barron空间$B_{\sigma}^{\varphi}$，其中下标$\sigma$表示激活函数，上标$\varphi$控制广义Barron空间的光滑性，通过施加于神经网络参数测度的$\varphi$加权积分范数定义。在$\varphi$和$\sigma$满足一定假设的条件下，证明$B_{\sigma}^{\varphi}$可连续嵌入到Sobolev空间中；还探究了各类浅层神经网络函数空间之间的关系，表明所提定义涵盖了大多数传统函数空间。作为广义Barron空间的应用，推导了这些空间内的逼近速率，并建立了以新引入的广义Barron范数为正则项的数值微分误差界。数值例子证实，所提空间允许在使用相同激活函数的情况下，构建具有不同光滑度的神经网络。
英文摘要：
Classical Barron spaces are function spaces specifically designed for shallow neural networks mostly with ReLU, $\mathrm{ReLU}^k$ (RePU) or Lipschitz continuous activation functions. In the present work, we introduce a generalized Barron space \( B_σ^φ \) for shallow neural networks with a generic activation function possessing certain smoothness properties. The subscript \( σ\) denotes the activation function, while the superscript \( φ\) controls the smoothness of the generalized Barron spaces, defined via a \( φ\)-weighted integral norm imposed on neural network parameter measures. Under certain assumptions on \( φ\) and \( σ\), we show that \( B_σ^φ \) can be continuously embedded into Sobolev spaces. We also explore the relationships among various function spaces for shallow neural networks, demonstrating that our definition encompasses most conventional ones. As applications of the proposed generalized Barron spaces, we derive approximation rates within these spaces and establish error bounds for numerical differentiation with regularization penalized by the newly introduced generalized Barron norm. Numerical examples confirm that the proposed spaces allow the construction of neural networks with varying degrees of smoothness while using the same activation function.
