arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.00901stat.MLcs.LGcs.NAmath.NA
收藏

关于使用深度ReLU神经网络对Sobolev和Besov函数的最优逼近

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks

School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science(数学学院(珠海)和广东省计算科学重点实验室)

Yunfei Yang

AI总结:

本文证明在Sobolev嵌入条件下,宽度为W、深度为L的深度ReLU神经网络逼近Sobolev和Besov函数可达到最优速率$\mathcal{O}((WL)^{-2s/d})$,关键贡献是提出一种新的稀疏向量深度网络编码方法。

摘要:

本文研究Sobolev空间$\mathcal{W}^{s,q}([0,1]^d)$和Besov空间$\mathcal{B}^s_{q,r}([0,1]^d)$中的函数在$L^p([0,1]^d)$范数度量误差下,能被宽度为$W$、深度为$L$的深度ReLU神经网络多有效地逼近的问题。该问题已被若干近期工作研究,它们在$p=q=\infty$时得到了忽略对数因子的逼近速率$\mathcal{O}((WL)^{-2s/d})$,并在Sobolev嵌入条件$1/q -1/p<s/d$成立时,对固定宽度网络得到了速率$\mathcal{O}(L^{-2s/d})$。我们推广了这些结果,证明在Sobolev嵌入条件下速率$\mathcal{O}((WL)^{-2s/d})$确实成立。已知该速率在忽略对数因子的意义下是最优的。我们证明中的关键工具是利用具有变化宽度和深度的深度ReLU神经网络对稀疏向量进行的一种新颖编码,这可能具有独立的研究意义。

英文摘要:

This paper studies the problem of how efficiently functions in the Sobolev spaces $\mathcal{W}^{s,q}([0,1]^d)$ and Besov spaces $\mathcal{B}^s_{q,r}([0,1]^d)$ can be approximated by deep ReLU neural networks with width $W$ and depth $L$, when the error is measured in the $L^p([0,1]^d)$ norm. This problem has been studied by several recent works, which obtained the approximation rate $\mathcal{O}((WL)^{-2s/d})$ up to logarithmic factors when $p=q=\infty$, and the rate $\mathcal{O}(L^{-2s/d})$ for networks with fixed width when the Sobolev embedding condition $1/q -1/p<s/d$ holds. We generalize these results by showing that the rate $\mathcal{O}((WL)^{-2s/d})$ indeed holds under the Sobolev embedding condition. It is known that this rate is optimal up to logarithmic factors. The key tool in our proof is a novel encoding of sparse vectors by using deep ReLU neural networks with varied width and depth, which may be of independent interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑