AI 中文总结
本文研究Dropout ReLU网络对Sobolev函数类的逼近，给出网络规模的上界与下界，并证明在固定或对数深度下精度指数匹配，揭示了随机保留边对逼近能力的影响。
AI 中文摘要
Dropout神经网络的通用逼近性质本身并不能描述实现精确随机实现所需的网络规模。本文研究了由ReLU网络对$W^{n,\infty}([0,1]^d)$单位球的逼近，其中网络的边以概率$p$独立保留。逼近误差在输入域上均匀度量，且保证以至少$1-\delta$的概率对单个采样网络成立。我们构造了深度恒定、规模为$\widetilde O_{n,d}(p^{-9}\varepsilon^{-\max\{d/n,2\}} \log(1/\delta))$的网络。该构造结合了有界局部子网络、在成功逼近事件上的局部化以及多尺度泰勒分解。相反，Sobolev容量对存活边的数量施加了下界，而对固定仿射函数的逼近在足够高的置信度下需要输出层代价为$((1-p)/p)\varepsilon^{-2}\log(1/\delta)$的量级。对于固定的$p\in(0,1)$和$\delta<\min\{1/2,1-p\}$，在固定或对数深度预算下，上界和下界在精度指数上匹配。当$d\leq2n$时，它们在置信度上匹配至精度的对数因子。我们将下界扩展到$W^{n,r}$目标与$L^s$误差，并区分此扩展与$W^{n,\infty}$的上界。最优保留依赖性和对数因子仍为开放问题。
英文摘要
The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realization. In this work, we study approximation of the unit ball of $W^{n,\infty}([0,1]^d)$ by ReLU networks whose edges are retained independently with probability $p$. The approximation error is measured uniformly over the input domain, and the guarantee holds with probability at least $1-δ$ for a single sampled network. We construct networks of constant depth and size $\widetilde O_{n,d}(p^{-9}\varepsilon^{-\max\{d/n,2\}} \log(1/δ))$. The construction combines bounded local subnetworks, localization on a successful approximation event, and a multiscale Taylor decomposition. Conversely, Sobolev capacity imposes a lower bound on the number of surviving edges, while approximation of a fixed affine function requires an output-layer cost of order $((1-p)/p)\varepsilon^{-2}\log(1/δ)$ at sufficiently high confidence. For fixed $p\in(0,1)$ and $δ<\min\{1/2,1-p\}$, the upper and lower bounds match in the accuracy exponent under a fixed or logarithmic depth budget. When $d\leq2n$, they also match in confidence up to logarithms of accuracy. We extend the lower bounds to $W^{n,r}$ targets with $L^s$ error, and distinguish this extension from the upper bound for $W^{n,\infty}$. The optimal retention dependence and logarithmic factors remain open.
Comments30 pages, 1 figure
