神经网络动力学的计算机验证：一个初步案例研究
Computer Validation of Neural Network Dynamics: A First Case Study
AI总结：
本文提出基于验证数值方法的范式，利用计算机辅助证明在循环神经网络中验证了几百个Hopf分岔点和周期轨道的存在性，为解释机器学习方法的复杂非线性动力学行为提供了初步途径。
摘要：
当前大量的机器学习方法依赖于深度神经网络。然而，将神经网络视为非线性动力系统时，很快就会发现，在数学上严格证明网络节点生成的某些模式是极其困难的。事实上，在复杂系统的非线性动力学中，人们已经充分认识到，即使在低维模型中，基于纸笔方法的分析技术也常常达到其极限。在这项工作中，我们通过非线性动力学的验证数值方法范式提出了一种完全不同的视角。其思想是使用计算机辅助证明来在数学上验证神经网络中非线性模式的存在性。作为一个案例研究，我们考虑了一类循环神经网络（RNN），在其中我们通过计算机辅助证明了几百个Hopf分岔点的存在性、它们的非退化性，从而也证明了几百个周期轨道的存在性。我们的范式具有严格验证神经网络复杂非线性行为的能力，这为通过计算机辅助证明解释机器学习方法的全部能力以及潜在敏感性提供了第一步。我们展示了验证数值技术如何能够阐明循环神经网络（RNNs）的内部工作机制。为此，Hopf分岔的证明是通过在各种设置下证明周期轨道的存在性，将动力系统理论整合到RNNs实际应用中的第一步。
英文摘要：
A large number of current machine learning methods rely upon deep neural networks. Yet, viewing neural networks as nonlinear dynamical systems, it becomes quickly apparent that mathematically rigorously establishing certain patterns generated by the nodes in the network is extremely difficult. Indeed, it is well-understood in the nonlinear dynamics of complex systems that, even in low-dimensional models, analytical techniques rooted in pencil-and-paper approaches frequently reach their limits. In this work, we propose a completely different perspective via the paradigm of validated numerical methods of nonlinear dynamics. The idea is to use computer-assisted proofs to validate mathematically the existence of nonlinear patterns in neural networks. As a case study, we consider a class of recurrent neural networks, where we prove via computer assistance the existence of several hundred Hopf bifurcation points, their non-degeneracy, and hence also the existence of several hundred periodic orbits. Our paradigm has the capability to rigorously verify complex nonlinear behaviour of neural networks, which provides a first step to explain the full abilities, as well as potential sensitivities, of machine learning methods via computer-assisted proofs. We showcase how validated numerical techniques can shed light on the internal working of recurrent neural networks (RNNs). For this, proofs of Hopf bifurcations are a first step towards an integration of dynamical system theory in practical application of RNNs, by proving the existence of periodic orbits in a variety of settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
