arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.04347q-bio.NC
收藏

具有强化作用的定时控制神经网络模型

A neural network model for timing control with reinforcement

Jing Wang, Yousuf El-Jayyousi, Ilker Ozden

摘要:

当可能性的空间是无限的时候,人类和动物是如何进行试错学习的?在以前的研究中,我们使用了一个间隔定时生产任务,并发现了一个更新策略,其中代理调整的行为和神经元的噪音探索。在实验中,人类受试者主动产生一系列定时运动输出。我们发现,顺序电机时序变化在两个时间尺度:长期相关的目标间隔,由于记忆漂移和短期调整的时间变异性,根据反馈。我们之前已经描述了这些功能的时间变化与增广高斯过程,称为奖励敏感高斯过程(RSGP)。在这里,我们提供了一个机械模型,并通过借用递归神经网络的架构来模拟这个过程。虽然经常性连接提供了运动计时的长期序列相关性,但为了促进奖励驱动的短期变化,我们在网络连接中引入了奖励依赖性变异性,这是受到大脑突触传递的随机性的启发。我们的模型能够递归地生成一个输出序列,将内部变异和外部强化的贝叶斯框架。我们证明了该模型可以学习人类行为的关键特征。与其他神经网络模型不同的是,该模型可以估计与每个结果相关的不确定性,从而更好地将可调整的任务相关变异性与无法解释的变异性区分开来。所提出的人工神经网络模型与神经系统中的信息处理机制并行,可以扩展连续状态控制中的脑启发强化学习框架。

英文摘要:

How do humans and animals perform trial-and-error learning when the space of possibilities is infinite? In a previous study, we used an interval timing production task and discovered an updating strategy in which the agent adjusted the behavioral and neuronal noise for exploration. In the experiment, human subjects proactively generated a series of timed motor outputs. We found that the sequential motor timing varied at two temporal scales: long-term correlation around the target interval due to memory drifts and short-term adjustments of timing variability according to feedback. We have previously described these features of timing variability with an augmented Gaussian process, termed reward sensitive Gaussian process (RSGP). Here we provide a mechanistic model and simulate the process by borrowing the architecture of recurrent neural networks. While recurrent connection provided the long-term serial correlation in motor timing, to facilitate reward-driven short-term variations, we introduced reward-dependent variability in the network connectivity, inspired by the stochastic nature of synaptic transmission in the brain. Our model was able to recursively generate an output sequence incorporating the internal variability and external reinforcement in a Bayesian framework. We show that the model can learn the key features of human behavior. Unlike other neural network models that search for unique network connectivity for the best match between the model prediction and observation, this model can estimate the uncertainty associated with each outcome and thus did a better job in teasing apart adjustable task-relevant variability from unexplained variability. The proposed artificial neural network model parallels the mechanisms of information processing in neural systems and can extend the framework of brain-inspired reinforcement learning in continuous state control.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑