发表机构
University of South Dakota（南达科他大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出非平衡动力学平均场理论，揭示循环神经网络在学习中从混沌到稳定的动力学转变，并定量预测训练过程输出演化。
AI 中文摘要
循环神经网络中的学习能够从根本上重塑其底层动力学，将初始混沌的活动转变为稳定的任务依赖行为。我们发展了一种非平衡动力学平均场理论（DMFT）来描述学习过程中的这一转变。我们表明，一个缓慢的反馈驱动学习过程会产生一个演化的有效反馈强度，该强度驱动网络经历从混沌到稳定动力学的转变，这一转变由DMFT解的分岔定义。通过推导整个学习过程中的双时间关联函数，我们识别出一个临界反馈强度以及相应的依赖于学习速率的临界时间，这些量将这两个区域分开。该转变源于不断增长的已学习反馈结构对有效动力学景观的渐进变形。从未训练状态出发，该理论预测了训练期间网络输出的时间演化，并与数值模拟显示出定量的一致性。
英文摘要
Learning in recurrent neural networks can fundamentally reshape their underlying dynamics, transforming initially chaotic activity into stable task-dependent behavior. We develop a non-equilibrium dynamical mean-field theory(DMFT) to describe this transition during learning. We show that a slow feedback-driven learning process generates an evolving effective feedback strength that drives the network through a transition from chaotic to stable dynamics defined by a bifurcation of the DMFT solution. By deriving the two-time correlation function throughout learning, we identify a critical feedback strength and a corresponding learning rate dependent critical time separating these regimes. The transition arises from the progressive deformation of an effective dynamical landscape by the growing learned feedback structure. Starting from the untrained state, the theory predicts the time evolution of the network output during training and shows quantitative agreement with numerical simulations.
Comments16 pages, 7 figures
