arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.10583cs.AIcs.LG
收藏

强化学习的三大教条

Three Dogmas of Reinforcement Learning

Google DeepMind(谷歌DeepMind);New York University(纽约大学)

David Abel, Mark K. Ho, Anna Harutyunyan

AI总结:

该研究指出强化学习领域存在环境聚焦、任务求解式学习观、奖励假设三大教条,提出应彻底摒弃前两条、以细致方式对待第三条,以释放强化学习作为智能体研究标准框架的潜力。

摘要:

现代强化学习至少受到三大教条的制约。第一条是“环境聚光灯”,指我们倾向于聚焦于环境建模而非智能体建模。第二条是我们将学习视为寻找任务的解决方案,而非适应过程。第三条是奖励假设,即所有目标和意图都可以被很好地理解为对奖励信号的最大化。这三大教条在很大程度上塑造了我们所认知的强化学习科学。尽管每条教条都在该领域的发展中发挥了重要作用,但现在是时候将它们摆上台面,反思它们是否应作为我们科学范式的基本组成部分。为了充分发挥强化学习作为研究智能体的标准框架的潜力,我们认为现在应当彻底摒弃第一和第二条教条,并以更细致的方式对待第三条教条。

英文摘要:

Modern reinforcement learning has been conditioned by at least three dogmas. The first is the environment spotlight, which refers to our tendency to focus on modeling environments rather than agents. The second is our treatment of learning as finding the solution to a task, rather than adaptation. The third is the reward hypothesis, which states that all goals and purposes can be well thought of as maximization of a reward signal. These three dogmas shape much of what we think of as the science of reinforcement learning. While each of the dogmas have played an important role in developing the field, it is time we bring them to the surface and reflect on whether they belong as basic ingredients of our scientific paradigm. In order to realize the potential of reinforcement learning as a canonical frame for researching intelligent agents, we suggest that it is time we shed dogmas one and two entirely, and embrace a nuanced approach to the third.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑