arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.07436cs.AI
收藏

考虑人类代理在马尔可夫决策过程执行期间不确定性影响的计算策略

Computing Policies That Account For The Effects Of Human Agent Uncertainty During Execution In Markov Decision Processes

Arizona State University(亚利桑那州立大学)

Sriram Gopalakrishnan, Mudit Verma, Subbarao Kambhampati

摘要:

当人类被赋予一个策略来执行时,如果在识别状态时存在不确定性,则可能存在策略执行错误和策略偏差。这可能是由于人类代理的认知限制和/或感知错误而发生的。因此,计算供人类执行的策略的算法应该在计算中考虑这些影响。一个最优的马尔可夫决策过程(MDP)政策,执行不佳(由于人类代理)可能比另一个政策是次优的MDP,但考虑人类代理的执行行为。在本文中,我们考虑两个问题,从状态的不确定性,这些是错误的状态推断,和额外的感知行动,一个人可能会采取由于他们的不确定性。我们提出了一个框架来模拟人类代理的行为与状态的不确定性,并可用于计算MDP政策,占这些问题。其次是一个爬山算法,以寻找良好的政策,我们的模型的人类代理。我们还提出了一个分支定界算法,可以找到这样的问题的最优策略。我们展示了在Gridworld域和仓库工人域中的实验结果。最后,我们提出了支持我们的人类模型假设的人类受试者研究。

英文摘要:

When humans are given a policy to execute, there can be policy execution errors and deviations in policy if there is uncertainty in identifying a state. This can happen due to the human agent's cognitive limitations and/or perceptual errors. So an algorithm that computes a policy for a human to execute ought to consider these effects in its computations. An optimal Markov Decision Process (MDP) policy that is poorly executed (because of a human agent) maybe much worse than another policy that is suboptimal in the MDP, but considers the human-agent's execution behavior. In this paper we consider two problems that arise from state uncertainty; these are erroneous state-inference, and extra-sensing actions that a person might take as a result of their uncertainty. We present a framework to model the human agent's behavior with respect to state uncertainty, and can be used to compute MDP policies that accounts for these problems. This is followed by a hill climbing algorithm to search for good policies given our model of the human agent. We also present a branch and bound algorithm which can find the optimal policy for such problems. We show experimental results in a Gridworld domain, and warehouse-worker domain. Finally, we present human-subject studies that support our human model assumptions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑