arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14344cs.LGcs.AI
收藏

通过结构化LTL表示在强化学习中实现零样本指令跟随

Zero-Shot Instruction Following in RL via Structured LTL Representations

Department of Computer Science, University of Oxford(牛津大学计算机科学系);Oxford Robotics Institute, University of Oxford(牛津大学机器人研究所)

Mathias Jackermeier, Mattia Giuri, Jacques Cloete, Alessandro Abate

AI总结:

本文提出了一种基于结构化LTL表示的强化学习方法,通过分层神经网络和注意力机制提升智能体在未见过任务中的指令跟随能力。

摘要:

我们研究了多任务强化学习中的指令跟随问题,其中智能体必须在未见过的训练任务中执行新的任务。在这一设定中,线性时序逻辑(LTL)最近被采用为一种强大的框架,用于指定结构化、时间延续的任务。尽管现有方法成功地训练了通用策略,但它们往往难以有效捕捉LTL规范中固有的丰富逻辑和时间结构。在本工作中,我们提出了一种新的方法来学习结构化任务表示,以促进训练和泛化。我们的方法将策略条件于由有限自动机构造的布尔公式序列。我们提出了一种分层神经网络架构来编码这些公式的逻辑结构,并引入一种注意力机制,使策略能够推理解未来子目标。在各种复杂环境中的实验展示了我们方法强大的泛化能力和优越的性能。

英文摘要:

We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training. In this setting, linear temporal logic (LTL) has recently been adopted as a powerful framework for specifying structured, temporally extended tasks. While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in LTL specifications. In this work, we address these concerns with a novel approach to learn structured task representations that facilitate training and generalisation. Our method conditions the policy on sequences of Boolean formulae constructed from a finite automaton of the task. We propose a hierarchical neural architecture to encode the logical structure of these formulae, and introduce an attention mechanism that enables the policy to reason about future subgoals. Experiments in a variety of complex environments demonstrate the strong generalisation capabilities and superior performance of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑