arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18299cs.AI
收藏

无模型RL智能体展现出类似系统1的意向性

Model-Free RL Agents Demonstrate System 1-Like Intentionality

University College London(伦敦大学学院)

Hal Ashton, Matija Franklin

AI总结:

本文提出一个新框架,将无模型RL智能体的反应行为类比为人类认知系统1,论证其无需显式规划即可展现意向性,并探讨该视角对AI责任归咎与安全监管的跨学科启示。

摘要:

本文论证了无模型强化学习(RL)智能体虽然缺乏显式规划机制,却表现出可类比为人类认知中系统1(“快思考”)过程的行为。基于模型的RL智能体通过利用内部表征进行规划,其运作类似于系统2(“慢思考”)推理;与之不同,无模型智能体对环境刺激作出反应,而不进行预测性建模。我们提出了一个新颖框架,将系统1与系统2的二元对立与无模型和基于模型的RL之间的区别联系起来。该框架挑战了意向性和目的性行为需要规划这一普遍假设,提出意向性可以体现在无模型智能体结构化的反应行为中。通过借鉴认知心理学、法学理论和实验法学的跨学科见解,我们探讨了该视角对归责和确保AI安全的影响。这些见解提倡对RL系统中的意向性进行更广泛、结合语境的解读,对其伦理部署与监管具有启示意义。

英文摘要:

This paper argues that model-free reinforcement learning (RL) agents, while lacking explicit planning mechanisms, exhibit behaviours that can be analogised to System 1 ("thinking fast") processes in human cognition. Unlike model-based RL agents, which operate akin to System 2 ("thinking slow") reasoning by leveraging internal representations for planning, model-free agents react to environmental stimuli without anticipatory modelling. We propose a novel framework linking the dichotomy of System 1 and System 2 to the distinction between model-free and model-based RL. This framing challenges the prevailing assumption that intentionality and purposeful behaviour require planning, suggesting instead that intentionality can manifest in the structured, reactive behaviours of model-free agents. By drawing on interdisciplinary insights from cognitive psychology, legal theory, and experimental jurisprudence, we explore the implications of this perspective for attributing responsibility and ensuring AI safety. These insights advocate for a broader, contextually informed interpretation of intentionality in RL systems, with implications for their ethical deployment and regulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑