用简化模型最小化规划的负面影响
Minimizing the Negative Side Effects of Planning with Reduced Models
摘要：
大型马尔可夫决策过程的简化模型通过考虑每个状态-动作对的结果子集来加速规划。当代理在计划执行期间遇到没有预先计算的操作的状态时，可达状态的这种减少导致重新计划。然而，并非所有国家都适合重新规划。在最坏的情况下，智能体可能无法从新遇到的状态到达目标。代理人应该更好地准备处理这种风险情况，避免在风险状态下重新规划。因此，我们认为在不安全的州重新规划是减少模型规划的负面影响。虽然可以通过始终使用完整模型来最小化负面影响，但这违背了使用简化模型的目的。挑战在于使用简化的模型进行计划，但不知何故考虑到遇到风险情况的可能性。因此，代理应该仅在用户已批准为安全的状态中进行重新规划。为此，我们建议规划使用组合的简化模型，规划范式，最大限度地减少规划使用简化模型的负面影响，通过交替不同的结果选择方法。我们实证证明了我们的方法在三个领域的有效性：电动汽车充电领域使用来自大学校园的真实数据和两个基准规划问题。
英文摘要：
Reduced models of large Markov decision processes accelerate planning by considering a subset of outcomes for each state-action pair. This reduction in reachable states leads to replanning when the agent encounters states without a precomputed action during plan execution. However, not all states are suitable for replanning. In the worst case, the agent may not be able to reach the goal from the newly encountered state. Agents should be better prepared to handle such risky situations and avoid replanning in risky states. Hence, we consider replanning in states that are unsafe for deliberation as a negative side effect of planning with reduced models. While the negative side effects can be minimized by always using the full model, this defeats the purpose of using reduced models. The challenge is to plan with reduced models, but somehow account for the possibility of encountering risky situations. An agent should thus only replan in states that the user has approved as safe for replanning. To that end, we propose planning using a portfolio of reduced models, a planning paradigm that minimizes the negative side effects of planning using reduced models by alternating between different outcome selection approaches. We empirically demonstrate the effectiveness of our approach on three domains: an electric vehicle charging domain using real-world data from a university campus and two benchmark planning problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
