arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.13754cs.AIcs.MAcs.RO
收藏

在不确定环境中规划时提升信息的价值

Increasing the Value of Information During Planning in Uncertain Environments

Oberlin College(欧柏林学院)

Gaurab Pokharel

AI总结:

针对信息收集与使用存在时间延迟导致信息价值被忽视的问题,提出在POMCP算法的UCB1启发式中加入Entropy的新算法,显著提升了在线规划性能。

摘要:

先前的研究表明,对于许多现实世界的问题,POMDPs可以通过在线算法快速且接近最优地求解。然而,在一类重要的问题中,即智能体收集信息的时间与其需要使用该信息的时间之间存在较大延迟时,这些解决方案未能充分考虑信息的价值。因此,信息收集动作,即使在最优策略中是关键的,也会被现有解决方案忽略,导致智能体做出次优决策。在本研究中,我们开发了一种新的解决方案,通过引入一种新算法来纠正这一问题,该算法通过更好地反思收集信息的动作的价值来改进最先进的在线规划。我们通过在POMCP算法的UCB1启发式中加入Entropy来实现这一点。我们在走廊问题上测试了该解决方案。结果表明,我们的新算法性能显著优于POMCP。

英文摘要:

Prior studies have demonstrated that for many real-world problems, POMDPs can be solved through online algorithms both quickly and with near optimality. However, on an important set of problems where there is a large time delay between when the agent can gather information and when it needs to use that information, these solutions fail to adequately consider the value of information. As a result, information gathering actions, even when they are critical in the optimal policy, will be ignored by existing solutions, leading to sub-optimal decisions by the agent. In this research, we develop a novel solution that rectifies this problem by introducing a new algorithm that improves upon state-of-the-art online planning by better reflecting on the value of actions that gather information. We do this by adding Entropy to the UCB1 heuristic in the POMCP algorithm. We test this solution on the hallway problem. Results indicate that our new algorithm performs significantly better than POMCP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑