Key-Gram: 用于具身操作的可扩展世界知识
Key-Gram: Extensible World Knowledge for Embodied Manipulation
AI总结：
本文提出Key-Gram框架，通过分离语言知识与视觉状态推理，提升具身控制中对组合语言指令的理解和执行能力，主要贡献是引入可扩展的外部记忆模块以提高迁移和现实世界操作性能。
摘要：
具身控制越来越多地要求模型在动态视觉状态上进行推理的同时遵循组合语言指令。然而，当前的视觉-语言-动作策略和世界-动作模型通常将语言知识与视觉计算结合在共享的骨干或条件路径中，导致模态竞争，并使知识扩展依赖于骨干更新。在本文中，我们引入了Key-Gram，一种条件记忆框架，它将语言衍生的世界知识与视觉状态推理分离用于具身控制。其核心是一个记忆模块，该模块将指令分解为任务特定的关键词组，通过确定性哈希查找检索静态语言先验，并通过上下文感知门控和轻量级卷积融合将检索到的条目注入到选定的隐藏层中。这种设计使骨干能够将其主要能力用于视觉推理和动作推断，同时可重用的指令知识存储在可扩展的外部记忆中。逻辑记忆表可以在训练期间方便地划分，并且由于其O(1)的查找模式，在推理时可以高效地放置在主机内存中。在RoboTwin2.0、LIBERO/LIBERO-Plus和现实世界双臂操作中，Key-Gram一致地提高了π₀和π₀.₅骨干，平均相对增益为RoboTwin2.0上的29.5%/9.9%、LIBERO-Plus转移无目标领域微调时的35.8%/4.5%以及现实世界长周期任务上的15.4%/8.1%。这些结果表明，外部化的语言记忆提供了一种有效的、可扩展的机制，以提高组合基础、迁移和现实世界操作性能。
英文摘要：
Embodied control increasingly requires models to follow compositional language instructions while reasoning over dynamic visual states. However, current vision-language-action policies and world-action models often couple linguistic knowledge with visual computation in a shared backbone or conditioning pathway, leading to modality competition and making knowledge extension dependent on backbone updates. In this paper, we introduce Key-Gram, a conditional-memory framework that separates language-derived world knowledge from visual-state reasoning for embodied control. At its core is a memory module that decomposes an instruction into task-specific key-grams, retrieves static linguistic priors through deterministic hashed lookup, and injects the retrieved entries into selected hidden layers through context-aware gating and lightweight convolutional fusion. This design allows the backbone to devote its main capacity to visual reasoning and action inference, while reusable instruction knowledge is stored in an extensible external memory. The logical memory table can be conveniently partitioned during training and, due to its $O(1)$ lookup pattern, efficiently placed on host memory during inference. Across RoboTwin2.0, LIBERO/LIBERO-Plus, and real-world dual-arm manipulation, Key-Gram consistently improves both $π_{0}$ and $π_{0.5}$ backbones, with average relative gains of $29.5\%/9.9\%$ on RoboTwin2.0, $35.8\%/4.5\%$ on LIBERO-Plus transfer without target-domain fine-tuning, and $15.4\%/8.1\%$ on real-world long-horizon tasks. These results demonstrate that externalized linguistic memory provides an effective and extensible mechanism for improving compositional grounding, transfer, and real-world manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
