arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.09181cs.LGcs.ITcs.NAmath.ITmath.NA
收藏

用一位神经网络逼近函数

Approximation of functions with one-bit neural networks

Courant Institute, New York University(纽约大学柯朗研究所);City University of New York, City College(纽约城市大学城市学院)

C. Sinan Güntürk, Weilin Li

摘要:

著名的神经网络的通用逼近定理粗略地说,任何合理的函数可以被一个网络任意地很好地逼近,该网络的参数是适当选择的实数。本文研究了一位神经网络的逼近能力-那些非零参数是$\pm a$的一些固定的$a\not=0$。我们的一个主要定理表明,对于任何具有$\|f\|_\infty<1$和错误$\varepsilon$的$f\in C^s([0,1]^d)$,存在一个$f_{NN}$,使得对于远离$[0,1]^d$边界的所有$\boldsymbol{x}$,$|f(\boldsymbol{x})-f_{NN}(\boldsymbol{x})|\leq \varepsilon$,并且$f_{NN}$可以由具有$O(\varepsilon^{-2d/s})$参数的$\{\pm 1\}$二次网络或具有$O(\varepsilon^{-2d/s}\log (1/\varepsilon))$参数的$\{\pm \frac 1 2 \}$ ReLU网络实现,如$\varepsilon\to0$。我们建立了迭代多变量Bernstein算子的新近似结果,Bernstein基础上噪声整形量化的误差估计,以及一位二次和ReLU神经网络的Bernstein多项式的新实现。

英文摘要:

The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the approximation capabilities of one-bit neural networks -- those whose nonzero parameters are $\pm a$ for some fixed $a\not=0$. One of our main theorems shows that for any $f\in C^s([0,1]^d)$ with $\|f\|_\infty<1$ and error $\varepsilon$, there is a $f_{NN}$ such that $|f(\boldsymbol{x})-f_{NN}(\boldsymbol{x})|\leq \varepsilon$ for all $\boldsymbol{x}$ away from the boundary of $[0,1]^d$, and $f_{NN}$ is either implementable by a $\{\pm 1\}$ quadratic network with $O(\varepsilon^{-2d/s})$ parameters or a $\{\pm \frac 1 2 \}$ ReLU network with $O(\varepsilon^{-2d/s}\log (1/\varepsilon))$ parameters, as $\varepsilon\to0$. We establish new approximation results for iterated multivariate Bernstein operators, error estimates for noise-shaping quantization on the Bernstein basis, and novel implementation of the Bernstein polynomials by one-bit quadratic and ReLU neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑