arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.10855cs.LGcs.RO
收藏

强化学习与选项及状态表示

Reinforcement Learning with Options and State Representation

RIKEN AIP(理化学研究所人工智能项目);ENS Paris-Saclay(巴黎萨克雷高等师范学校)

Ayoub Ghriss, Masashi Sugiyama, Alessandro Lazaric

AI总结:

本文针对高维复杂环境的学习问题,提出基于分层强化学习的框架,通过管理者与员工智能体构建分层策略,并引入基于环境图结构的Eigenoption,使智能体决策对对称变换不变,从而大幅降低学习复杂度。

摘要:

本论文旨在探索强化学习领域,并在现有方法的基础上进行改进,以解决高维复杂环境中的学习问题。它通过以分层方式分解学习任务来实现这些目标,这种方式被称为分层强化学习。在第一章中,我们首先熟悉马尔可夫决策过程框架,并介绍后续章节将使用的一些近期技术。随后,我们着手构建分层策略学习,作为对单一原始策略局限性的解决方案。该层级结构由顶部的管理者智能体和底层的员工智能体组成。在作为本文核心的最后一章中,我们尝试独立于管理者层级来学习层级结构的底层元素,即所谓的“Eigenoption”。基于环境的图结构,Eigenoptions 使我们能够构建感知环境几何与动态特性的智能体。其决策具有一个特殊属性:对环境的对称变换保持不变性,从而大大降低了学习任务的复杂度。

英文摘要:

The current thesis aims to explore the reinforcement learning field and build on existing methods to produce improved ones to tackle the problem of learning in high-dimensional and complex environments. It addresses such goals by decomposing learning tasks in a hierarchical fashion known as Hierarchical Reinforcement Learning. We start in the first chapter by getting familiar with the Markov Decision Process framework and presenting some of its recent techniques that the following chapters use. We then proceed to build our Hierarchical Policy learning as an answer to the limitations of a single primitive policy. The hierarchy is composed of a manager agent at the top and employee agents at the lower level. In the last chapter, which is the core of this thesis, we attempt to learn lower-level elements of the hierarchy independently of the manager level in what is known as the "Eigenoption". Based on the graph structure of the environment, Eigenoptions allow us to build agents that are aware of the geometric and dynamic properties of the environment. Their decision-making has a special property: it is invariant to symmetric transformations of the environment, allowing as a consequence to greatly reduce the complexity of the learning task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑