arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1609.07672cs.LG
收藏

部分可观测马尔可夫决策过程规划与学习的信息论方法

Information-Theoretic Methods for Planning and Learning in Partially Observable Markov Decision Processes

Hebrew University of Jerusalem(耶路撒冷希伯来大学)

Roy Fox

摘要:

有界智能体在处理其传感器和记忆中可用的信息以及选择动作和记忆更新的能力上受到内在约束的限制。在这篇论文中,我们将这些约束建模为连接这些代理内部组件的通信信道上的信息速率约束。我们做了四个主要的贡献,下面详细介绍了每个部分中详细介绍的许多较小的贡献。首先,我们制定的问题,优化代理下的外在和内在的约束和开发的主要工具来解决它。其次,我们确定的另一个原因,具有挑战性的收敛性能的优化算法,这是分支结构的更新算子附近的相变。第三,我们研究了线性高斯动态和二次成本(LQG)的特殊情况下,最优解有一个特别简单和可解的形式。第四,我们探索学习任务,其中世界动态模型是未知的,而是使用基于样本的更新。

英文摘要:

Bounded agents are limited by intrinsic constraints on their ability to process information that is available in their sensors and memory and choose actions and memory updates. In this dissertation, we model these constraints as information-rate constraints on communication channels connecting these various internal components of the agent. We make four major contributions detailed below and many smaller contributions detailed in each section. First, we formulate the problem of optimizing the agent under both extrinsic and intrinsic constraints and develop the main tools for solving it. Second, we identify another reason for the challenging convergence properties of the optimization algorithm, which is the bifurcation structure of the update operator near phase transitions. Third, we study the special case of linear-Gaussian dynamics and quadratic cost (LQG), where the optimal solution has a particularly simple and solvable form. Fourth, we explore the learning task, where the model of the world dynamics is unknown and sample-based updates are used instead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑