arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1801.03737cs.AI
收藏

POMDPs的反事实等价性和潜在的确定性环境

Counterfactual equivalence for POMDPs, and underlying deterministic environments

Future of Humanity Institute(人类未来研究所);University of Oxford(牛津大学);Machine Intelligence Research Institute(机器智能研究所)

Stuart Armstrong

摘要:

部分可观测马尔可夫决策过程(POMDP)是机器学习中经常使用的丰富环境。但是POMDPs中的信息和因果结构问题却相对较少被研究。本文提出了等效和反事实等效POMDP的概念,其中代理人无法通过任何观察和动作区分他们所处的环境。它表明,任何POMDP是反事实等价的,对于任何有限的匝数,一个确定性POMDP与所有的不确定性集中到初始状态。这允许更好地理解POMDP的不确定性、信息和学习。

英文摘要:

Partially Observable Markov Decision Processes (POMDPs) are rich environments often used in machine learning. But the issue of information and causal structures in POMDPs has been relatively little studied. This paper presents the concepts of equivalent and counterfactually equivalent POMDPs, where agents cannot distinguish which environment they are in though any observations and actions. It shows that any POMDP is counterfactually equivalent, for any finite number of turns, to a deterministic POMDP with all uncertainty concentrated into the initial state. This allows a better understanding of POMDP uncertainty, information, and learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑