arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.07080math.NAcs.NAmath.PRstat.ML
收藏

高维深度学习:$L^2(\mathbb{R}^d,γ_d)$中解析函数的神经网络逼近

Deep Learning in High Dimension: Neural Network Approximation of Analytic Functions in $L^2(\mathbb{R}^d,γ_d)$

Christoph Schwab, Jakob Zech

摘要:

对于人工深度神经网络,我们证明了解析函数$f:\mathbb{R}^d\to\mathbb{R}$在$L^2(\mathbb{R}^d,γ_d)$范数下的表达率,其中$d\in {\mathbb{N}}\cup\{ \infty \}$。这里$γ_d$表示$\mathbb{R}^d$上的高斯乘积概率测度。我们特别考虑整数$k\geq 2$的ReLU和ReLU ${}^k$激活。对于$d\in\mathbb{N}$,我们证明了$L^2(\mathbb{R}^d,γ_d)$的指数收敛速度。在$d=\infty$的情况下,在适当的光滑性和稀疏性假设下,$f:\mathbb{R}^{\mathbb{N}}\to\mathbb{R}$,与$γ_\infty$表示一个无限(高斯)的产品措施$\mathbb{R}^{\mathbb{N}}$,我们证明了维数无关的表达率界的规范$L^2(\mathbb{R}^{\mathbb{N}},γ_\infty)$。该速率仅取决于$f$到$\mathbb{C}^d$中的条带的乘积的映射(的解析延拓)的量化全纯性。作为应用,我们证明了对数高斯随机场输入的椭圆偏微分方程响应面的深度ReLU-NN的表达率界。

英文摘要:

For artificial deep neural networks, we prove expression rates for analytic functions $f:\mathbb{R}^d\to\mathbb{R}$ in the norm of $L^2(\mathbb{R}^d,γ_d)$ where $d\in {\mathbb{N}}\cup\{ \infty \}$. Here $γ_d$ denotes the Gaussian product probability measure on $\mathbb{R}^d$. We consider in particular ReLU and ReLU${}^k$ activations for integer $k\geq 2$. For $d\in\mathbb{N}$, we show exponential convergence rates in $L^2(\mathbb{R}^d,γ_d)$. In case $d=\infty$, under suitable smoothness and sparsity assumptions on $f:\mathbb{R}^{\mathbb{N}}\to\mathbb{R}$, with $γ_\infty$ denoting an infinite (Gaussian) product measure on $\mathbb{R}^{\mathbb{N}}$, we prove dimension-independent expression rate bounds in the norm of $L^2(\mathbb{R}^{\mathbb{N}},γ_\infty)$. The rates only depend on quantified holomorphy of (an analytic continuation of) the map $f$ to a product of strips in $\mathbb{C}^d$. As an application, we prove expression rate bounds of deep ReLU-NNs for response surfaces of elliptic PDEs with log-Gaussian random field inputs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑