arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.06005cs.AIcs.LGmath.OC
收藏

代理空间

Agent Spaces

Massachusetts General Hospital(麻省总医院)

John C. Raisbeck, Matthew W. Allen, Hakho Lee

摘要:

探索是强化学习中最重要的任务之一,但在动态规划范式中,它并没有很好地定义为有限问题(见2.4小节)。我们提供了一个可以应用于任何在线学习方法的探索的重新解释。我们从一个新的方向来探索这个定义。在发现探索的概念,以解决简单的马尔可夫决策过程与动态规划不再广泛适用,我们重新审视探索。我们不是扩展动态探索过程的目的,而是扩展它们的手段。也就是说,而不是重复采样每一个可能的状态动作对在一个过程中,我们定义的行为修改代理本身是探索性的。由此产生的探索定义可以应用于无限的问题和非动态学习方法,这是动态探索概念所不能容忍的。为了理解代理的修改影响学习的方式,我们描述了代理集合上的一种新结构:距离的集合(见脚注7)$d_{a} \in A$,它代表了过程中每个代理可能的视角。使用这些距离,我们定义了一个拓扑结构,并表明强化学习中的许多重要结构在智能体空间中的收敛引起的拓扑结构下表现良好。

英文摘要:

Exploration is one of the most important tasks in Reinforcement Learning, but it is not well-defined beyond finite problems in the Dynamic Programming paradigm (see Subsection 2.4). We provide a reinterpretation of exploration which can be applied to any online learning method. We come to this definition by approaching exploration from a new direction. After finding that concepts of exploration created to solve simple Markov decision processes with Dynamic Programming are no longer broadly applicable, we reexamine exploration. Instead of extending the ends of dynamic exploration procedures, we extend their means. That is, rather than repeatedly sampling every state-action pair possible in a process, we define the act of modifying an agent to itself be explorative. The resulting definition of exploration can be applied in infinite problems and non-dynamic learning methods, which the dynamic notion of exploration cannot tolerate. To understand the way that modifications of an agent affect learning, we describe a novel structure on the set of agents: a collection of distances (see footnote 7) $d_{a} \in A$, which represent the perspectives of each agent possible in the process. Using these distances, we define a topology and show that many important structures in Reinforcement Learning are well behaved under the topology induced by convergence in the agent space.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑