arXiv 2603.129422026-03-13cs.RO

ReMem-VLA：通过双层递归查询增强视觉-语言-动作模型的记忆能力

ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries

Technical University of Munich（慕尼黑技术大学）；Huawei Heisenberg Research Center（华为海森堡研究中心）；University of Freiburg（弗赖堡大学）；Nanjing University（南京大学）；Huawei Technologies（华为技术）

Hang Li, Fengyi Shen, Dong Chen, Liudi Yang, Xudong Wang, Jinkui Shi, Zhenshan Bing, Ziyuan Liu, Alois Knoll

备注：: 14 pages, 6 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ReMem-VLA通过双层递归查询机制提升视觉-语言-动作模型的记忆能力，结合帧级和块级记忆查询，增强短期和长期记忆，通过端到端训练实现上下文聚合与维护，优于现有基线模型。

摘要：

视觉-语言-动作（VLA）模型在闭环机器人控制中通常基于马尔可夫假设，导致需要历史上下文的任务易出错。为引入记忆，现有VLA要么从记忆库检索，易受干扰；要么扩展帧窗口，固定时间跨度限制长期记忆。本文提出ReMem-VLA，一种配备双组可学习查询的递归记忆VLA模型：帧级递归记忆查询用于跨连续帧传播信息以支持短期记忆，块级递归记忆查询用于跨时间块携带上下文以支持长期记忆。这些查询通过端到端训练聚合和维护相关上下文，隐式指导模型决策，无需额外训练或推理成本。此外，为增强视觉记忆，引入过去观察预测作为辅助训练目标。通过广泛的记忆导向仿真和现实机器人实验，证明ReMem-VLA在多个维度（空间、序列、事件、时间、视觉记忆）均表现出强大的记忆能力。ReMem-VLA显著优于无记忆VLA基线模型π0.5和OpenVLA-OFT，并在依赖记忆的任务上大幅超越MemoryVLA。

英文摘要：

Vision-language-action (VLA) models for closed-loop robot control are typically cast under the Markov assumption, making them prone to errors on tasks requiring historical context. To incorporate memory, existing VLAs either retrieve from a memory bank, which can be misled by distractors, or extend the frame window, whose fixed horizon still limits long-term retention. In this paper, we introduce ReMem-VLA, a Recurrent Memory VLA model equipped with two sets of learnable queries: frame-level recurrent memory queries for propagating information across consecutive frames to support short-term memory, and chunk-level recurrent memory queries for carrying context across temporal chunks for long-term memory. These queries are trained end-to-end to aggregate and maintain relevant context over time, implicitly guiding the model's decisions without additional training or inference cost. Furthermore, to enhance visual memory, we introduce Past Observation Prediction as an auxiliary training objective. Through extensive memory-centric simulation and real-world robot experiments, we demonstrate that ReMem-VLA exhibits strong memory capabilities across multiple dimensions, including spatial, sequential, episodic, temporal, and visual memory. ReMem-VLA significantly outperforms memory-free VLA baselines $π$0.5 and OpenVLA-OFT and surpasses MemoryVLA on memory-dependent tasks by a large margin.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。