arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09169cs.LGcs.AI
收藏

变化环境中基于因子化状态表示学习鲁棒策略的课程设计

Curricula for Learning Robust Policies with Factored State Representations in Changing Environments

University of Bath(巴斯大学)

Panayiotis Panayiotou, Özgür Şimşek

AI总结:

本文研究因子化状态表示下课程设计对强化学习策略鲁棒性的影响,实验证明三种简单课程(如仅改变最高遗憾变量)可显著提升鲁棒性,为复杂环境提供实用指导。

摘要:

鲁棒策略使强化学习智能体能够有效适应并在不可预测、动态且不断变化的现实环境中运行。因子化表示将复杂的状态和动作空间分解为不同的组成部分,可以提高策略学习中的泛化能力和样本效率。在本文中,我们探讨了使用因子化状态表示的智能体的课程如何影响所学策略的鲁棒性。我们通过实验证明了三种简单课程(例如仅在回合之间改变最高遗憾变量)能够显著增强策略鲁棒性,为复杂环境中的强化学习提供了实用见解。

英文摘要:

Robust policies enable reinforcement learning agents to effectively adapt to and operate in unpredictable, dynamic, and ever-changing real-world environments. Factored representations, which break down complex state and action spaces into distinct components, can improve generalization and sample efficiency in policy learning. In this paper, we explore how the curriculum of an agent using a factored state representation affects the robustness of the learned policy. We experimentally demonstrate three simple curricula, such as varying only the variable of highest regret between episodes, that can significantly enhance policy robustness, offering practical insights for reinforcement learning in complex environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑