发表机构
UC San Diego; Northwestern University; Massachusetts Institute of Technology（加州大学圣地亚哥分校; 西北大学; 麻省理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于空间神经计算的二阶循环模型，将其等价为结构化无限阶RNN，通过约束梯度谱消除梯度问题，在长时序基准上性能优于其他循环模型且参数更少。
AI 中文摘要
循环神经网络（RNN）随序列长度呈线性时间扩展，且仅需恒定内存，但因梯度消失和感受野有限，难以捕获长程依赖关系。为解决这些局限，我们提出一种二阶循环模型，其中标准的神经元间通信被由（离散化）偏微分方程控制的空间演化场取代。受大脑计算中皮层波作用的启发，该机制允许结构化时空模式作为隐式、高容量记忆。我们证明所得模型等价于结构化无限阶RNN，其中当前状态显式依赖于其全部过去状态历史，在参数数量固定的情况下产生有效无界的感受野。我们进一步推导构造性条件以确保边际稳定性，将梯度谱约束在单位圆上，从而消除梯度消失与梯度爆炸。实验表明，所提架构在长时序基准测试上优于其他循环模型，同时使用的参数显著更少，证明空间动力学可有效弥合高效推理与长期记忆之间的差距。
英文摘要
Recurrent neural networks (RNNs) offer linear-time scaling with sequence length while requiring only constant memory, yet they struggle to capture long-range dependencies due to vanishing gradients and limited receptive fields. To address these limitations, we introduce a second-order recurrent model in which the standard neuron-to-neuron communication is replaced by a spatially evolving field governed by (discretized) partial differential equations. Drawing inspiration from the role of cortical waves in brain computation, this mechanism allows structured spatiotemporal patterns to serve as an implicit, high-capacity memory. We show that the resulting model is equivalent to a structured infinite-order RNN in which the current state depends explicitly on its entire history of past states, yielding an effectively unbounded receptive field with a fixed number of parameters. We further derive constructive conditions to ensure marginal stability, constraining the gradient spectrum on the unit circle and thereby eliminating vanishing and exploding gradients. Empirically, the proposed architecture outperforms other recurrent models on long-horizon benchmarks while using substantially fewer parameters, demonstrating that spatial dynamics can effectively bridge the gap between efficient inference and long-term memory.
Comments12 pages, 4 figures
