arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08355cs.AI
收藏

ASPECT:通过语言条件转移实现类比语义策略执行

ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer

Applied Artificial Intelligence Initiative, Deakin University, Waurn Ponds, Geelong(迪肯大学应用人工智能计划);School of IT, Deakin University, Waurn Ponds, Geelong(迪肯大学信息技术学院)

Ajsal Shereef Palattuparambil, Thommen George Karimpanal, Santu Rana

AI总结:

ASPECT通过语言条件化的变分自编码器和大语言模型实现零样本迁移,解决传统方法在新任务适应性差的问题。

摘要:

强化学习(RL)代理往往难以将知识泛化到新任务,即使这些任务在结构上与已掌握的任务相似。尽管近期方法试图通过零样本迁移缓解这一问题,但它们通常受限于预定义的离散类别系统,限制了对新或组合式任务变化的适应能力。我们提出了一种显著更通用的方法,用自然语言条件替代离散潜在变量,通过文本条件化的变分自编码器(VAE)。我们的核心创新是在测试时利用大型语言模型(LLM)作为动态的语义操作符。而非依赖刚性规则,我们的代理会查询LLM,将当前观察的描述语义地映射到源任务,使生成的描述与源任务对齐,从而引导VAE生成与代理原始训练兼容的想象状态,实现直接的策略重用。通过利用LLM灵活的推理能力,我们的方法在广泛复杂的真正新颖类比任务上实现了零样本迁移,超越了固定类别映射的限制。代码和视频可在 \href{https://anonymous.4open.science/r/ASPECT-85C3/}{此处} 获取。

英文摘要:

Reinforcement Learning (RL) agents often struggle to generalize knowledge to new tasks, even those structurally similar to ones they have mastered. Although recent approaches have attempted to mitigate this issue via zero-shot transfer, they are often constrained by predefined, discrete class systems, limiting their adaptability to novel or compositional task variations. We propose a significantly more generalized approach, replacing discrete latent variables with natural language conditioning via a text-conditioned Variational Autoencoder (VAE). Our core innovation utilizes a Large Language Model (LLM) as a dynamic \textit{semantic operator} at test time. Rather than relying on rigid rules, our agent queries the LLM to semantically remap the description of the current observation to align with the source task. This source-aligned caption conditions the VAE to generate an imagined state compatible with the agent's original training, enabling direct policy reuse. By harnessing the flexible reasoning capabilities of LLMs, our approach achieves zero-shot transfer across a broad spectrum of complex and truly novel analogous tasks, moving beyond the limitations of fixed category mappings. Code and videos are available \href{https://anonymous.4open.science/r/ASPECT-85C3/}{here}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑