arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.15820cs.AI
收藏

论部分可观测性下的浅层规划

On shallow planning under partial observability

Canada CIFAR AI Chair(加拿大高级研究院人工智能主席职位)

Randy Lefebvre, Audrey Durand

AI总结:

本文研究强化学习中折扣因子所表征的规划时域,在给定 MDP 结构参数下分析其对偏差—方差权衡的影响,指出较短规划时域在部分可观测环境中尤其可能带来收益。

摘要:

在强化学习框架下形式化现实问题涉及并非简单的设计选择,例如为学习目标(折扣累积奖励)选择折扣因子,该因子明确了智能体的规划时域。本研究在给定底层 Markov Decision Process 的结构参数时,考察折扣因子对偏差—方差权衡的影响。我们的结果支持如下观点:更短的规划时域可能是有益的,尤其是在部分可观测性下。

英文摘要:

Formulating a real-world problem under the Reinforcement Learning framework involves non-trivial design choices, such as selecting a discount factor for the learning objective (discounted cumulative rewards), which articulates the planning horizon of the agent. This work investigates the impact of the discount factor on the bias-variance trade-off given structural parameters of the underlying Markov Decision Process. Our results support the idea that a shorter planning horizon might be beneficial, especially under partial observability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑