用于循环神经网络机理解释的情景记忆理论
Episodic Memory Theory for the Mechanistic Interpretation of Recurrent Neural Networks
AI总结：
该研究提出用于解释RNN机理的情景记忆理论，通过设计变量绑定任务验证RNN会收敛到对应回路，并提出特权基算法提升RNN可解释性，为理解RNN机制和衔接人工神经网络与神经记忆模型提供了新路径。
摘要：
从机理层面理解循环神经网络（RNN）的复杂运作，对提升其性能与拓展应用至关重要。为此，我们提出情景记忆理论（Episodic Memory Theory, EMT），阐明RNN可被视为新近提出的通用序列情景记忆模型的离散时间类比。为验证EMT，我们设计了一组全新的算法任务，专门用于探究RNN中的变量绑定行为。借助EMT，我们构建了一个数学上严谨的回路，可在这些任务中实现变量绑定。实证研究表明，经过训练的RNN会一致收敛到该变量绑定回路，这揭示了RNN动力学的普遍性。基于这些发现，我们设计了一种定义特权基的算法，该基能揭示在变量的时序存储与组合中起关键作用的隐藏神经元，而这一机制是这些任务中实现成功泛化的核心。我们证明，特权基提升了RNN所学习参数与隐藏状态的可解释性。本工作向揭开RNN内部机制的神秘面纱迈出了一步，同时在计算神经科学领域，为人工神经网络与神经记忆模型搭建了桥梁。
英文摘要：
Understanding the intricate operations of Recurrent Neural Networks (RNNs) mechanistically is pivotal for advancing their capabilities and applications. In this pursuit, we propose the Episodic Memory Theory (EMT), illustrating that RNNs can be conceptualized as discrete-time analogs of the recently proposed General Sequential Episodic Memory Model. To substantiate EMT, we introduce a novel set of algorithmic tasks tailored to probe the variable binding behavior in RNNs. Utilizing the EMT, we formulate a mathematically rigorous circuit that facilitates variable binding in these tasks. Our empirical investigations reveal that trained RNNs consistently converge to the variable binding circuit, thus indicating universality in the dynamics of RNNs. Building on these findings, we devise an algorithm to define a privileged basis, which reveals hidden neurons instrumental in the temporal storage and composition of variables, a mechanism vital for the successful generalization in these tasks. We show that the privileged basis enhances the interpretability of the learned parameters and hidden states of RNNs. Our work represents a step toward demystifying the internal mechanisms of RNNs and, for computational neuroscience, serves to bridge the gap between artificial neural networks and neural memory models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
