arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.07647cs.RO
收藏

TempoFit: 无需训练的分层时间KV内存用于长时视觉-语言-动作操控

TempoFit: Plug-and-Play Layer-Wise Temporal KV Memory for Long-Horizon Vision-Language-Action Manipulation

Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学)

Jun Sun, Boyu Yang, Jiahao Zhang, Ning Ma, Chencheng Wu, Siqing Zhang, Yiou Huang, Qiufeng Wang, Shan Liang, Yaran Chen

AI总结:

TempoFit通过无需训练的分层时间KV内存提升长时视觉-语言-动作操控性能,有效解决记忆缺失问题,提升任务成功率。

摘要:

预训练的视觉-语言-动作(VLA)策略在单步操控方面取得了显著成果,但其推理过程仍主要依赖于记忆,这在非马尔可夫性长时程场景中显得脆弱,尤其是在存在遮挡、状态别名和细微动作后变化的情况下。先前的方法通过堆叠帧来注入历史,这会扩大视觉令牌并增加延迟,同时引入近似重复的像素;或者通过学习额外的时间接口,这需要(重新)训练,可能会破坏原始单帧推理图。我们提出了TempoFit,一种无需训练的时间 retrofit,通过状态层面的内存升级冻结的VLA。我们的关键见解是,前缀注意力K/V已经形成了模型原生的内容寻址运行时状态;在不同时间步之间重用它们可以引入历史而不增加新的令牌或可训练模块。TempoFit在选定的中间层存储分层FIFO前缀K/V,通过帧间隙时间偏置(FGTB),一种受NLP中位置偏置启发的固定近期偏置,进行参数无关的K到K检索,以保持决策的当前主导性,并通过预注意力残差加载和规范保持的重缩放将检索到的上下文注入,以避免在冻结权重下出现分布偏移。在LIBERO-LONG上,TempoFit在保持接近实时延迟的同时,将强大的预训练骨干网络的平均成功率提高了最高4.0%,并且能够一致地转移到CALVIN和真实机器人长时程任务。

英文摘要:

Pretrained Vision-Language-Action (VLA) policies have achieved strong single-step manipulation, but their inference remains largely memoryless, which is brittle in non-Markovian long-horizon settings with occlusion, state aliasing, and subtle post-action changes. Prior approaches inject history either by stacking frames, which scales visual tokens and latency while adding near-duplicate pixels, or by learning additional temporal interfaces that require (re-)training and may break the original single-frame inference graph. We present TempoFit, a training-free temporal retrofit that upgrades frozen VLAs through state-level memory. Our key insight is that prefix attention K/V already form a model-native, content-addressable runtime state; reusing them across timesteps introduces history without new tokens or trainable modules. TempoFit stores layer-wise FIFO prefix K/V at selected intermediate layers, performs parameter-free K-to-K retrieval with Frame-Gap Temporal Bias (FGTB), a fixed recency bias inspired by positional biases in NLP, to keep decisions present-dominant, and injects the retrieved context via pre-attention residual loading with norm-preserving rescaling to avoid distribution shift under frozen weights. On LIBERO-LONG, TempoFit improves strong pretrained backbones by up to +4.0% average success rate while maintaining near-real-time latency, and it transfers consistently to CALVIN and real-robot long-horizon tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑