分离任务与行为：一种强化学习中的两阶段奖励课程
Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
AI总结：
本文提出了一种两阶段奖励课程方法，通过分离任务目标与行为术语，提升机器人强化学习中奖励函数设计的有效性和稳定性。
摘要：
深度强化学习是一种有前景的机器人控制工具，但实际应用往往受到设计有效奖励函数的困难所阻碍。现实任务通常需要同时优化多个目标，这需要精确调整其权重以学习具有所需特性的策略。为此，我们提出了一种两阶段奖励课程，其中我们将任务特定的目标与行为术语分离。在我们的方法中，我们首先在简化任务-only奖励函数上训练智能体以确保有效的探索，然后再引入包含辅助行为相关术语（如能耗效率）的完整奖励。进一步，我们分析了各种转移策略，并证明在阶段之间重用样本对训练稳定性至关重要。我们在DeepMind Control Suite、ManiSkill3和一个包含辅助行为目标的移动机器人环境中验证了我们的方法。我们的方法证明是简单而有效的，显著优于直接在完整奖励上训练的基线方法，同时表现出更高的对特定奖励权重的鲁棒性。
英文摘要：
Deep Reinforcement Learning is a promising tool for robotic control, yet practical application is often hindered by the difficulty of designing effective reward functions. Real-world tasks typically require optimizing multiple objectives simultaneously, necessitating precise tuning of their weights to learn a policy with the desired characteristics. To address this, we propose a two-stage reward curriculum where we decouple task-specific objectives from behavioral terms. In our method, we first train the agent on a simplified task-only reward function to ensure effective exploration before introducing the full reward that includes auxiliary behavior-related terms such as energy efficiency. Further, we analyze various transition strategies and demonstrate that reusing samples between phases is critical for training stability. We validate our approach on the DeepMind Control Suite, ManiSkill3, and a mobile robot environment, modified to include auxiliary behavioral objectives. Our method proves to be simple yet effective, substantially outperforming baselines trained directly on the full reward while exhibiting higher robustness to specific reward weightings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
