发表机构
Capital University of Economics and Business; Sun Yat-sen University（首都经济贸易大学; 中山大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究将ReLU神经网络的逼近结果从一致范数推广到Sobolev范数，针对浅层和深层网络分别给出了在路径范数约束下的逼近误差界，并去除了深层网络对光滑性的限制。
AI 中文摘要
近期研究表明，光滑函数可以通过具有路径范数约束权重的ReLU神经网络得到很好的逼近。我们将这些结果从一致逼近推广到Sobolev范数下的逼近。具体而言，我们分析了当逼近误差以$W^{1,p}$-范数度量时，$W^{n,p}$中的Sobolev函数能被宽度为$W$、深度为$L$且路径范数有界为$K$的神经网络逼近的程度。对于深度$L=1$的浅层网络，当光滑性指标满足$n<s=(d+3)/2$且输入为$d$维时，我们推导出逼近误差界为$\mathcal{O}(\max\{W^{-(n-1)/d}, K^{-(n-1)/(s-n)}\})$。对于深层网络，我们通过证明当宽度$W$和深度$L$足够大时，逼近界$\mathcal{O}(K^{-(n-1)/(d+d/p+1)})$成立，从而去除了对光滑性的限制。
英文摘要
Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in $W^{n,p}$ can be approximated by neural networks with width $W$, depth $L$ and path norm bounded by $K$, when the approximation error is measured in the $W^{1,p}$-norm. For shallow networks with depth $L=1$, we derive the approximation error bound $\mathcal{O}(\max\{W^{-(n-1)/d}, K^{-(n-1)/(s-n)}\})$, when the smoothness index satisfies $n<s=(d+3)/2$ and the input is $d$-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound $\mathcal{O}(K^{-(n-1)/(d+d/p+1)})$ holds if the width $W$ and depth $L$ are sufficiently large.
