arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.20166cs.RO
收藏

LoLA:面向通用机器人操作的长周期隐式动作学习

LoLA: Long Horizon Latent Action Learning for General Robot Manipulation

Institute of Microelectronics, Chinese Academy of Sciences(中国科学院微电子研究所);University of Chinese Academy of Sciences(中国科学院大学);Microsoft Research(微软研究院)

Xiaofan Wang, Xingyu Gao, Jianlong Fu, Zuolei Li, Dean Fortier, Galen Mullins, Andrey Kolobov, Baining Guo

AI总结:

LoLA通过整合长期多视角观察和机器人本体感觉,实现长周期、语言引导的机器人操作任务,显著优于现有方法。

摘要:

执行长周期、语言引导的机器人操作任务的能力关键依赖于利用历史信息并生成连贯的动作序列。然而,现有视觉-语言-动作(VLA)模型往往忽视了这种能力。为解决这一挑战,我们提出了LoLA(长周期隐式动作学习),一个专为机器人操作设计的框架,整合了长期多视角观察和机器人本体感觉,以实现多步骤推理和动作生成。我们首先使用视觉-语言模型对历史序列和多视角观察中的丰富上下文特征进行编码。我们进一步引入了一个关键模块,状态感知的隐式再表示,将视觉输入和语言指令转换为可执行的机器人运动空间。与现有VLA方法仅将机器人本体感觉(如关节角度)与VL嵌入拼接不同,该模块利用此类机器人状态,通过可学习的“具身体验锚定”隐空间,显式地将VL表示锚定在物理尺度上。我们训练LoLA在多样化的机器人预训练数据集上,并在模拟基准(SIMPLER和LIBERO)以及Franka和Bi-Manual Aloha机器人上的两个现实任务上进行了广泛评估。结果表明,LoLA在长周期操作任务中显著优于现有最先进的方法(如pi0)

英文摘要:

The capability of performing long-horizon, language-guided robotic manipulation tasks critically relies on leveraging historical information and generating coherent action sequences. However, such capabilities are often overlooked by existing Vision-Language-Action (VLA) models. To solve this challenge, we propose LoLA (Long Horizon Latent Action Learning), a framework designed for robot manipulation that integrates long-term multi-view observations and robot proprioception to enable multi-step reasoning and action generation. We first employ Vision-Language Models to encode rich contextual features from historical sequences and multi-view observations. We further introduces a key module, State-Aware Latent Re-representation, which transforms visual inputs and language commands into actionable robot motion space. Unlike existing VLA approaches that merely concatenate robot proprioception (e.g., joint angles) with VL embeddings, this module leverages such robot states to explicitly ground VL representations in physical scale through a learnable "embodiment-anchored" latent space. We trained LoLA on diverse robotic pre-training datasets and conducted extensive evaluations on simulation benchmarks (SIMPLER and LIBERO), as well as two real-world tasks on Franka and Bi-Manual Aloha robots. Results show that LoLA significantly outperforms prior state-of-the-art methods (e.g., pi0), particularly in long-horizon manipulation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑