用于万能逼近的动力学系统最小控制族
A Minimal Control Family of Dynamical Systems for Universal Approximation
AI总结:
本文证明仅含仿射映射和ReLU非线性映射的最小控制族可生成逼近保向微分同胚的流映射,揭示神经网络逼近能力与控制系统的联系。
摘要:
万能逼近性质(UAP)在深度学习中占据基础性地位,因为它为神经网络的表达能力提供了理论基础。众所周知,线性函数与非线性函数(如修正线性单元(ReLU)激活函数)的复合可以在紧致域上逼近连续函数。在本文中,我们将这一有效性扩展到包含控制的动力学系统场景中。我们证明了包含所有仿射映射和非线性ReLU映射的控制族$\mathcal{F}_1$足以生成能在任意紧致域上逼近保向(OP)微分同胚的流映射。由于$\mathcal{F}_1$仅包含一个非线性函数,且若移除该非线性函数则UAP不再成立,因此我们称$\mathcal{F}_1$为UAP的最小控制族。在此基础上,我们为控制族建立了若干温和的充分条件(如仿射不变性)并进行了讨论。我们的结果揭示了神经网络逼近能力与控制系统之间的内在联系,并为检验基于流的模型的逼近能力提供了理论指导。
英文摘要:
The universal approximation property (UAP) holds a fundamental position in deep learning, as it provides a theoretical foundation for the expressive power of neural networks. It is widely recognized that a composition of linear and nonlinear functions, such as the rectified linear unit (ReLU) activation function, can approximate continuous functions on compact domains. In this paper, we extend this efficacy to a scenario containing dynamical systems with controls. We prove that the control family $\mathcal{F}_1$ containing all affine maps and the nonlinear ReLU map is sufficient for generating flow maps that can approximate orientation-preserving (OP) diffeomorphisms on any compact domain. Since $\mathcal{F}_1$ contains only one nonlinear function and the UAP does not hold if we remove the nonlinear function, we call $\mathcal{F}_1$ a minimal control family for the UAP. On this basis, several mild sufficient conditions, such as affine invariance, are established for the control family and discussed. Our results reveal an underlying connection between the approximation power of neural networks and control systems and could provide theoretical guidance for examining the approximation power of flow-based models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。