神经振荡器的上近似界限
Upper Approximation Bounds for Neural Oscillators
AI总结:
本文研究了由二阶常微分方程和多层感知机组成的神经振荡器,推导了其在近似因果连续算子和二阶动力系统中的上近似界限,并验证了收敛速率。
摘要:
神经振荡器起源于二阶常微分方程,已在稳定学习长期序列或连续时间函数的因果映射以及准确近似物理系统方面表现出色。然而,理论量化其神经网络架构的容量仍是一个重大挑战。本文研究了由二阶ODE后接多层感知机(MLP)构成的神经振荡器,推导了其在近似连续时间函数空间间的因果和一致连续算子以及近似一致渐进增量稳定的二阶动力系统中的上近似界限。所建立的近似界限证明方法也可直接应用于由线性时间连续复杂递归神经网络后接MLP构成的状态空间模型。理论结果表明,神经振荡器近似二阶动力系统的近似误差与所用MLP宽度的倒数呈多项式关系,从而克服了参数复杂度的诅咒。通过四个数值案例验证了两个建立的近似误差界限的收敛速率。这些结果为神经振荡器在科学和工程中的有效应用提供了稳健的理论基础。
英文摘要:
Neural oscillators, originating from second-order ordinary differential equations (ODEs), have demonstrated strong performance in stably learning causal mappings between long-term sequences or continuous temporal functions, as well as in accurately approximating physical systems. However, theoretically quantifying the capacities of their neural network architectures remains a significant challenge. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper approximation bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating uniformly asymptotically incrementally stable second-order dynamical systems are derived. The established proof method of the approximation bound for approximating the causal continuous operators can also be directly applied to state-space models consisting of a linear time-continuous complex recurrent neural network followed by an MLP. Theoretical results reveal that the approximation error of the neural oscillator for approximating the second-order dynamical systems scales polynomially with the reciprocals of the widths of two utilized MLPs, thus overcoming the curse of parametric complexity. The convergence rates of two established approximation error bounds are validated through four numerical cases. These results provide a robust theoretical foundation for the effective application of the neural oscillator in science and engineering.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。