深度神经网络的逼近能力：一篇解释性数学综述
Approximation Power of Deep Neural Networks: an explanatory mathematical survey
AI总结：
本综述系统解释深度神经网络（前馈与残差）的逼近性质，分析其优于传统方法的条件，涵盖优化训练、稠密性及误差-复杂度分析，并引入新理论结果。
摘要：
本综述深入且解释性地回顾了深度神经网络的逼近性质，重点关注前馈和残差架构。主要目标是考察神经网络如何有效地逼近目标函数，并确定它们优于传统逼近方法的条件。关键主题包括深度网络的非线性、组合结构，以及将神经网络任务形式化为回归和分类设置中的优化问题。综述还讨论了训练过程，强调随机梯度下降和反向传播在解决这些优化问题中的作用，并强调了激活函数、过拟合和正则化技术等实际考虑因素。此外，综述探讨了神经网络在连续函数空间中的稠密性，将深度ReLU网络的逼近能力与其他逼近方法进行了比较。它讨论了最近在理解这些网络的表现力和局限性方面的理论进展。还呈现了详细的误差-复杂度分析，重点关注在具有最小正则性假设的有界目标函数背景下，具有ReLU和傅里叶型激活函数的神经网络的误差率和计算复杂度。除了最近已知的结果外，综述还介绍了新发现，为理解神经网络逼近的理论基础提供了宝贵的资源。最后给出了结论性评论和进一步阅读建议。
英文摘要：
This survey provides an in-depth and explanatory review of the approximation properties of deep neural networks, with a focus on feed-forward and residual architectures. The primary objective is to examine how effectively neural networks approximate target functions and to identify conditions under which they outperform traditional approximation methods. Key topics include the nonlinear, compositional structure of deep networks and the formalization of neural network tasks as optimization problems in regression and classification settings. The survey also addresses the training process, emphasizing the role of stochastic gradient descent and backpropagation in solving these optimization problems, and highlights practical considerations such as activation functions, overfitting, and regularization techniques. Additionally, the survey explores the density of neural networks in the space of continuous functions, comparing the approximation capabilities of deep ReLU networks with those of other approximation methods. It discusses recent theoretical advancements in understanding the expressiveness and limitations of these networks. A detailed error-complexity analysis is also presented, focusing on error rates and computational complexity for neural networks with ReLU and Fourier-type activation functions in the context of bounded target functions with minimal regularity assumptions. Alongside recent known results, the survey introduces new findings, offering a valuable resource for understanding the theoretical foundations of neural network approximation. Concluding remarks and further reading suggestions are provided.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
