Robota-Embed：用于机器人上下文学习与进化的统一VLA嵌入
Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution
AI总结：
本研究提出首个联合编码视觉-语言-动作三模态的Robota-Embed模型，在Robota-Embed-Bench上性能优于Gemini Embedding 2，可支撑机器人上下文学习与任务局部进化，在多任务中提升了机器人任务成功率。
摘要：
为机器人检索有用上下文需要匹配其任务、执行阶段及后续应执行的操作。我们推出Robota-Embed，据我们所知，这是首个视觉-语言-动作（VLA）嵌入模型，可在当前状态查询和记录的未来延续中联合编码所有三种模态。预测检索训练对齐这些端点，使历史延续能在推理时作为上下文使用。我们构建了包含1,695,684个清洗后训练锚点的多源语料库，并推出Robota-Embed-Bench，其包含4,000个查询和共享的8,000条条目库。Robota-Embed在片段级实现47.33%的R@1，在回合级实现81.77%的R@1，比无动作的Gemini Embedding 2分别高出22.09和33.47个百分点。受控检索研究显示，移除原生动作对检索的损害大于移除视觉。我们研究了两个互补应用：Embed2ICL利用检索到的经验进行无训练和基于训练的策略条件调整，在DROID-Sim中成功率从47.0%提升至56.3%，在LIBERO困难子集上从86.4%提升至92.8%，同时提升了真实机器人的任务进度；Embed2Evolve利用执行反馈扩展任务局部记忆并优化冻结的GPT-6 Astra策略的上下文使用，在5项ManiSkill任务中，其成功率达68%，而GPT-6 Astra Direct为24%，在相同协议下用Omni-Embed-Nemotron-3B独立进化的工具为33%；与后者相比，其每次测试滚动（含成功与失败）使用的策略令牌减少约79.5%。这些结果表明，统一的、感知动作的检索空间可支持机器人上下文学习和任务局部上下文进化。
英文摘要：
Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware retrieval space can support robot in-context learning and task-local context evolution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
