效率与鲁棒性划分循环网络中记忆的解空间

中文摘要 AI 辅助

在计算神经科学中，任务训练的循环神经网络（RNNs）通常被用作探索与特定功能兼容的循环电路解空间的试验平台。然而，这些网络受到归纳偏置的影响，这些偏置可能与生物电路（其在各种效率和鲁棒性约束下运行）的偏置不一致。在此，我们使用一个最小刺激回忆任务，系统地刻画了循环神经网络的解空间如何被权重效率、活动效率和噪声鲁棒性这些目标所塑造。我们表明，权重高效的解表现出低秩偏置，这与任务训练网络的归纳偏置一致，并有利于涉及缓慢、持续动力学的动态模式，而活动效率则鼓励依赖于瞬态放大的高秩解。此外，我们发现活动高效的解在活动的高方差模式与因果影响网络输出的模式之间表现出强烈的分离。与静态前馈设置不同，我们证明噪声鲁棒性和活动效率通常是相互对立的，而非协同的。利用这些发现，我们提出了一个关于最近在前运动皮层中发现的错位编码现象的标准性解释。

英文摘要

In computational neuroscience, task-trained recurrent neural networks (RNNs) are commonly used as a testbed for exploring the space of recurrent circuit solutions compatible with a particular function. However, these networks are subject to inductive biases that may be misaligned with those of biological circuits, which operate under various efficiency and robustness constraints. Here, using a minimal stimulus recall task, we systematically characterize how the solution space of recurrent neural networks is shaped by the desiderata of weight efficiency, activity efficiency, and robustness to noise. We show that weight efficient solutions exhibit a low-rank bias that aligns with the inductive biases of task-trained networks and favors dynamical motifs involving slow, persistent dynamics, whereas activity efficiency encourages high-rank solutions that rely on transient amplification. Moreover, we find that activity efficient solutions exhibit a strong dissociation between high-variance modes of activity and those that causally influence network output. Unlike in static feedforward settings, we demonstrate that noise robustness and activity efficiency are generally in opposition, rather than synergistic. Using these findings, we propose a normative account of misaligned coding phenomena recently identified in premotor cortex.

发表机构

• Harvard University（哈佛大学）
• Kempner Institute for the Study of Natural and Artificial Intelligence（肯普纳自然与人工智能研究所）
• John A. Paulson School of Engineering and Applied Sciences（约翰·A·保尔森工程与应用科学学院）
• Center for Brain Science（脑科学中心）

机构由 AI 辅助整理，请以论文原文为准。