关于使用深度ReLU神经网络对Sobolev和Besov函数的最优逼近
On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks
AI总结:
本文证明在Sobolev嵌入条件下,宽度为W、深度为L的深度ReLU神经网络逼近Sobolev和Besov函数可达到最优速率$\mathcal{O}((WL)^{-2s/d})$,关键贡献是提出一种新的稀疏向量深度网络编码方法。
摘要:
本文研究Sobolev空间$\mathcal{W}^{s,q}([0,1]^d)$和Besov空间$\mathcal{B}^s_{q,r}([0,1]^d)$中的函数在$L^p([0,1]^d)$范数度量误差下,能被宽度为$W$、深度为$L$的深度ReLU神经网络多有效地逼近的问题。该问题已被若干近期工作研究,它们在$p=q=\infty$时得到了忽略对数因子的逼近速率$\mathcal{O}((WL)^{-2s/d})$,并在Sobolev嵌入条件$1/q -1/p<s/d$成立时,对固定宽度网络得到了速率$\mathcal{O}(L^{-2s/d})$。我们推广了这些结果,证明在Sobolev嵌入条件下速率$\mathcal{O}((WL)^{-2s/d})$确实成立。已知该速率在忽略对数因子的意义下是最优的。我们证明中的关键工具是利用具有变化宽度和深度的深度ReLU神经网络对稀疏向量进行的一种新颖编码,这可能具有独立的研究意义。
英文摘要:
This paper studies the problem of how efficiently functions in the Sobolev spaces $\mathcal{W}^{s,q}([0,1]^d)$ and Besov spaces $\mathcal{B}^s_{q,r}([0,1]^d)$ can be approximated by deep ReLU neural networks with width $W$ and depth $L$, when the error is measured in the $L^p([0,1]^d)$ norm. This problem has been studied by several recent works, which obtained the approximation rate $\mathcal{O}((WL)^{-2s/d})$ up to logarithmic factors when $p=q=\infty$, and the rate $\mathcal{O}(L^{-2s/d})$ for networks with fixed width when the Sobolev embedding condition $1/q -1/p<s/d$ holds. We generalize these results by showing that the rate $\mathcal{O}((WL)^{-2s/d})$ indeed holds under the Sobolev embedding condition. It is known that this rate is optimal up to logarithmic factors. The key tool in our proof is a novel encoding of sparse vectors by using deep ReLU neural networks with varied width and depth, which may be of independent interest.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。