A large body of work has suggested that neural populations exhibit low-dimensional dynamics during behavior. However, there are a variety of different approaches for modeling low-dimensional neural population activity. One approach involves latent linear dynamical system (LDS) models, in which population activity is described by a projection of low-dimensional latent variables with linear dynamics. A second approach involves low-rank recurrent neural networks (RNNs), in which population activity arises directly from a low-dimensional projection of past activity. Although these two modeling approaches have strong similarities, they arise in different contexts and tend to have different domains of application. Here we examine the precise relationship between latent LDS models and linear low-rank RNNs. When can one model class be converted to the other, and vice versa? We show that latent LDS models can only be converted to RNNs in specific limit cases, due to the non-Markovian property of latent LDS models. Conversely, we show that linear RNNs can be mapped onto LDS models, with latent dimensionality at most twice the rank of the RNN.
Efficiency and robustness partition the solution space for memory in recurrent networks
Harvard University（哈佛大学）；Kempner Institute for the Study of Natural and Artificial Intelligence（肯普纳自然与人工智能研究所）；John A. Paulson School of Engineering and Applied Sciences（约翰·A·保尔森工程与应用科学学院）；Center for Brain Science（脑科学中心）
Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory
School of System Science, Beijing Normal University（北京师范大学系统科学学院）；State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University（北京师范大学认知神经科学与学习国家重点实验室）；Qiyuan Laboratory（启元实验室）
Connectivity structure and dynamics of nonlinear recurrent neural networks
Zuckerman Institute, Columbia University（哥伦比亚大学祖克曼研究所）；Kavli Institute for Brain Science, Columbia University（哥伦比亚大学卡弗里脑科学研究所）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）
Peking University（北京大学）；IDG/McGovern Institute for Brain Research（IDG/麦戈文脑科学研究所）；Peking-Tsinghua Center for Life Sciences（北大-清华生命科学中心）；Academy for Advanced Interdisciplinary Studies, Peking University（北京大学前沿交叉学科研究院）；Center of Quantitative Biology, Peking University（北京大学定量生物学中心）
A ghost mechanism: An analytical model of abrupt learning in recurrent networks
Kavli Institute for Theoretical Physics, University of California, Santa Barbara, CA 93106, USA；Geometric Intelligence Lab, University of California, Santa Barbara, CA 93106, USA；CNC Program, Stanford University, Stanford, CA 94305, USA；Physics of Artificial Intelligence Group, NTT Research Inc., Sunnyvale, CA 94085, USA；KUIS AI, Department of Computer Engineering, Koc University, Istanbul, Turkey；Computer Science, Stanford University, Stanford, CA 94305, USA；James H. Clark Center for Biomedical Engineering \& Sciences, Stanford University, Stanford；Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA；CBS-NTT Program in Physics of Intelligence, Harvard University, Cambridge, MA 94305, USA
