arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.12402q-bio.NC
收藏

具有奖励预测误差的表示学习

Representation learning with reward prediction errors

William H. Alexander, Samuel J. Gershman

摘要:

奖励预测误差假说提出,中脑多巴胺能系统的阶段性活动反映了强化学习中学习所需的预测误差。除了多巴胺和奖励处理之间的关系有据可查之外,多巴胺还参与了各种功能,但与奖励预测错误之间没有明确的关系。多巴胺水平的波动影响对时间的主观感知,多巴胺爆发先于运动反应的产生,多巴胺能系统支配大脑的区域,包括海马体和前额叶皮层的区域,其功能并不是唯一与奖励有关的。在这篇手稿中,我们提出,一个共同的主题连接这些功能是代表性的,以及预测错误信号的多巴胺系统,除了驱动联想学习,也可以支持收购的自适应状态表示。在一系列的模拟中,我们展示了这种扩展如何解释多巴胺在时间和空间表征,运动反应和抽象分类任务中的作用。通过将多巴胺信号的作用扩展到学习状态表征,我们解决了对多巴胺功能的奖励预测误差假设的关键挑战。

英文摘要:

The Reward Prediction Error hypothesis proposes that phasic activity in the midbrain dopaminergic system reflects prediction errors needed for learning in reinforcement learning. Besides the well-documented association between dopamine and reward processing, dopamine is implicated in a variety of functions without a clear relationship to reward prediction error. Fluctuations in dopamine levels influence the subjective perception of time, dopamine bursts precede the generation of motor responses, and the dopaminergic system innervates regions of the brain, including hippocampus and areas in prefrontal cortex, whose function is not uniquely tied to reward. In this manuscript, we propose that a common theme linking these functions is representation, and that prediction errors signaled by the dopamine system, in addition to driving associative learning, can also support the acquisition of adaptive state representations. In a series of simulations, we show how this extension can account for the role of dopamine in temporal and spatial representation, motor response, and abstract categorization tasks. By extending the role of dopamine signals to learning state representations, we resolve a critical challenge to the Reward Prediction Error hypothesis of dopamine function.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑