arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06525cs.AI
收藏

行为树中强化学习的进展约束

Progress Constraints for Reinforcement Learning in Behavior Trees

Adaptive and Interpretable Learning Systems Lab, Department of Computer Science, Örebro University(奥雷布罗大学计算机科学系自适应与可解释学习系统实验室);Robotics, Perception and Learning Lab, School of Electrical Engineering and Computer Science, Royal Institute of Technology(皇家理工学院电气工程与计算机科学学院机器人、感知与学习实验室)

Finn Rietz, Mart Kartašev, Petter Ögren, Johannes A. Stork

AI总结:

本文提出进展约束机制,通过可行性估计器约束动作集,提升行为树与强化学习结合的性能、样本效率和约束满足度。

摘要:

行为树(BTs)提供了一种结构化且反应性的决策框架,通常用于根据环境条件在子控制器之间切换。强化学习(RL),另一方面,可以学习近最优控制器,但有时在稀疏奖励、安全探索和长时间信用分配方面遇到困难。将BT与RL结合有相互促进的潜力:一个BT设计编码了结构化的领域知识,可以简化RL训练,而RL则使BT中的控制器能够自动学习。然而,简单地将BT与RL结合可能导致某些控制器相互抵消,可能推翻之前达成的子目标,从而降低整体性能。为此,我们提出了进展约束,一种新的机制,其中可行性估计器根据理论BT收敛结果约束允许的动作集。在2D概念验证和高保真仓库环境中的实证评估显示,与先前的BT-RL结合方法相比,改进了性能、样本效率和约束满足度。

英文摘要:

Behavior Trees (BTs) provide a structured and reactive framework for decision-making, commonly used to switch between sub-controllers based on environmental conditions. Reinforcement Learning (RL), on the other hand, can learn near-optimal controllers but sometimes struggles with sparse rewards, safe exploration, and long-horizon credit assignment. Combining BTs with RL has the potential for mutual benefit: a BT design encodes structured domain knowledge that can simplify RL training, while RL enables automatic learning of the controllers within BTs. However, naive integration of BTs and RL can lead to some controllers counteracting other controllers, possibly undoing previously achieved subgoals, thereby degrading the overall performance. To address this, we propose progress constraints, a novel mechanism where feasibility estimators constrain the allowed action set based on theoretical BT convergence results. Empirical evaluations in a 2D proof-of-concept and a high-fidelity warehouse environment demonstrate improved performance, sample efficiency, and constraint satisfaction, compared to prior methods of BT-RL integration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑