arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.09356q-bio.NC
收藏

循环神经网络解空间的制图和导航

Charting and navigating the space of solutions for recurrent neural networks

Elia Turner, Kabir Dabholkar, Omri Barak

摘要:

最近,循环神经网络(RNN)被成功地用于模拟动物中神经活动驱动任务相关行为的方式,在所获得的解决方案是通用的隐含假设下操作。神经科学和机器学习的观察都对这一假设提出了挑战。动物可以用各种策略来完成给定的任务,而训练机器学习算法引入了规格不足的现象。这些观察意味着每个任务都与解决方案空间相关联。到目前为止,这个空间的结构还不清楚,限制了将RNN与神经数据进行比较的方法。在这里,我们描述了与各种任务相关的解决方案的空间。我们首先研究了一个简单的双神经元网络,该网络用于一个导致多个解决方案的任务。我们将最终解决方案的性质追溯到网络的初始连接性,并识别出这种多样性背后的离散动态机制。然后,我们研究了三个神经科学启发的任务:延迟和间隔歧视,时间再现。对于每个任务,我们都有丰富的解决方案。可变性可以直接在网络的神经活动中发现,并且另外通过测试训练的网络的外推能力来发现,因为对系统的扰动通常揭示隐藏的结构。此外,我们将外推模式与网络发现的特定动态对象和有效算法相关联。我们引入了一个工具,通过生成一个紧凑的有向图来描述关于行为输入和输出的动态本质,从而推导出网络的简化动态。使用这种表示法,我们可以将每个任务的解决方案划分为几种类型,并从神经特征中部分预测它们。我们的研究结果揭示了解决方案空间的概念及其在机器学习和神经科学中的应用。

英文摘要:

Recurrent Neural Networks (RNNs) were recently successfully used to model the way neural activity drives task-related behavior in animals, operating under the implicit assumption that the obtained solutions are universal. Observations in both neuroscience and machine learning challenge this assumption. Animals can approach a given task with a variety of strategies, and training machine learning algorithms introduces the phenomenon of underspecification. These observations imply that every task is associated with a space of solutions. To date, the structure of this space is not understood, limiting the approach of comparing RNNs with neural data. Here, we characterize the space of solutions associated with various tasks. We first study a simple two-neuron network on a task that leads to multiple solutions. We trace the nature of the final solution back to the network's initial connectivity and identify discrete dynamical regimes that underlie this diversity. We then examine three neuroscience-inspired tasks: Delayed and interval discrimination, and Time reproduction. For each task, we find a rich set of solutions. Variability can be found directly in the neural activity of the networks, and additionally by testing the trained networks' ability to extrapolate, as a perturbation to a system often reveals hidden structure. Furthermore, we relate extrapolation patterns to specific dynamical objects and effective algorithms found by the networks. We introduce a tool to derive the reduced dynamics of networks by generating a compact directed graph describing the essence of the dynamics with regards to behavioral inputs and outputs. Using this representation, we can partition the solutions to each task into a handful of types and partially predict them from neural features. Our results shed light on the concept of the space of solutions and its uses in Machine learning and in Neuroscience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑