发表机构
University of Edinburgh; Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT); NVIDIA; Robotics Institute Germany (RIG)（爱丁堡大学; 卡尔斯鲁厄理工学院直觉机器人实验室; 英伟达; 德国机器人研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
ReCAT 提出一种带结构化循环记忆的语言条件化策略，通过 Mamba-2 和因果注意力整合观测流，在 LIBERO 和 RMBench 上取得领先成功率，并在真实机器人任务中显著超越短历史基线，验证了记忆更新的不同规则对各类任务的影响。
AI 中文摘要
依赖记忆的操作要求机器人利用当前传感器不再可用的信息做出决策，例如回忆早期的视觉线索、跟踪任务进度、计数重复事件或估计经过的时间。我们提出了 ReCAT，一种具有结构化循环记忆的语言条件化策略。一个指令条件编码器从当前观测中形成特征。一个循环记忆通过 Mamba-2 层和一个因果注意力层整合观测流。一个流匹配 Transformer 解码器在每个块中通过独立的交叉注意力读取当前和历史表示。ReCAT 在 LIBERO 上达到 95.3% 的平均成功率，在 RMBench 上达到 62.4%，在九项任务中的六项上取得最佳或并列最佳结果。在三个探测空间回忆、事件计数和间隔计时的真实机器人任务上，最佳 ReCAT 变体达到 66.7% 的平均成功率，而最强的短历史基线仅为 8.3%。ReCAT 内部的受控比较表明，观测编码器和每个块的记忆条件化对于该性能是必需的。它们还表明，为高效序列建模而设计的更新规则在机器人记忆中的表现不同：加法更新在计数和计时上观察到最高的成功率，而 delta 规则更新在空间回忆上表现最佳。项目网站位于此 https URL。
英文摘要
Memory-dependent manipulation requires robots to make decisions using information that is no longer available to their current sensors, such as recalling an earlier visual cue, tracking task progress, counting repeated events, or estimating elapsed time. We present ReCAT, a language-conditioned policy with structured recurrent memory. An instruction-conditioned encoder forms features from the current observation. A recurrent memory integrates the observation stream through Mamba-2 layers and one causal attention layer. A flow-matching Transformer decoder reads the current and the historical representation through separate cross-attention in every block. ReCAT reaches 95.3\% average success on LIBERO and 62.4\% on RMBench, with the best or tied-best result on six of nine tasks. On three real-robot tasks probing spatial recall, event counting, and interval timing, the best ReCAT variant reaches 66.7\% average success, against 8.3\% for the strongest short-history baseline. Controlled comparisons within ReCAT show that the observation encoder and every-block memory conditioning are needed for this performance. They also show that update rules developed for efficient sequence modeling behave differently as robot memory: additive updates have the highest observed success on counting and timing, and delta-rule updates on spatial recall. Project website is at https://intuitive-robots.github.io/ReCAT
Comments9 pages, 3 figures
