arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01488cs.AI
收藏

基于大语言模型的语义选项发现用于促进自适应深度强化学习

LLM-assisted Semantic Option Discovery for Facilitating Adaptive Deep Reinforcement Learning

Chang Yao, Jinghui Qin, Kebing Jin, Hankz Hankui Zhuo

AI总结:

本文提出基于大语言模型的闭环框架,通过语义驱动技能重用和实时约束监控,提升深度强化学习在数据效率、约束合规性和跨任务迁移能力方面的性能。

摘要:

尽管在复杂任务中取得了显著成功,深度强化学习(DRL)在实际应用中仍然面临关键问题,例如低数据效率、缺乏可解释性以及有限的跨环境迁移能力。然而,通过状态生成动作的学得策略对环境变化敏感,难以保证行为安全性和合规性。最近的研究表明,将大语言模型(LLMs)与符号规划相结合在解决这些挑战方面具有前景。受此启发,我们引入了一个新的LLM驱动的闭环框架,该框架通过将自然语言指令映射到可执行规则并语义标注自动创建的选项,实现了语义驱动的技能重用和实时约束监控。所提出的方法利用LLMs的一般知识来促进探索效率,并适应于相似环境的可迁移选项,同时通过语义标注提供内在可解释性。为了验证该框架的有效性,我们在两个领域进行实验,分别是Office World和Montezuma's Revenge。结果表明,在数据效率、约束合规性和跨任务迁移能力方面表现优异。

英文摘要:

Despite achieving remarkable success in complex tasks, Deep Reinforcement Learning (DRL) is still suffering from critical issues in practical applications, such as low data efficiency, lack of interpretability, and limited cross-environment transferability. However, the learned policy generating actions based on states are sensitive to the environmental changes, struggling to guarantee behavioral safety and compliance. Recent research shows that integrating Large Language Models (LLMs) with symbolic planning is promising in addressing these challenges. Inspired by this, we introduce a novel LLM-driven closed-loop framework, which enables semantic-driven skill reuse and real-time constraint monitoring by mapping natural language instructions into executable rules and semantically annotating automatically created options. The proposed approach utilizes the general knowledge of LLMs to facilitate exploration efficiency and adapt to transferable options for similar environments, and provides inherent interpretability through semantic annotations. To validate the effectiveness of this framework, we conduct experiments on two domains, Office World and Montezuma's Revenge, respectively. The results demonstrate superior performance in data efficiency, constraint compliance, and cross-task transferability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑