模仿意图,而非仅仅轨迹
Mimic Intent, Not Just Trajectories
AI总结:
本文提出MINT方法,通过多尺度频域标记解耦行为意图与执行细节,提升模仿学习的适应性和迁移能力,实验显示其在任务基准和真实机器人上的优越性能。
摘要:
尽管模仿学习在灵巧操作中通过生成模型和预训练取得了显著成功,但最先进的方法如视觉-语言-动作(VLA)模型仍难以适应环境变化和技能迁移。我们提出MINT方法,通过多尺度频率空间标记解耦行为意图与执行细节,在端到端模仿学习中实现意图模仿而非单纯轨迹模仿。我们通过多尺度频率空间标记,强制动作块表示的频谱分解。我们学习动作标记采用多尺度粗到细结构,最粗的标记捕捉低频全局结构,更细的标记编码高频细节。这产生了一个抽象的意图标记,有助于规划和迁移,以及多尺度执行标记,使精确适应环境动态成为可能。基于此层次,我们的策略通过下一尺度自回归生成轨迹,进行逐步的意图到执行推理,从而提升学习效率和泛化能力。关键的是,这种解耦使技能的一次性迁移成为可能,只需将演示中的意图标记注入自回归生成过程。在多个操作基准和真实机器人上的实验显示,MINT在任务成功率、推理效率、抗干扰泛化能力和一次性迁移方面均取得最先进的成果。
英文摘要:
While imitation learning (IL) has achieved impressive success in dexterous manipulation through generative modeling and pretraining, state-of-the-art approaches like Vision-Language-Action (VLA) models still struggle with adaptation to environmental changes and skill transfer. We argue this stems from mimicking raw trajectories without understanding the underlying intent. To address this, we propose explicitly disentangling behavior intent from execution details in end-2-end IL: Mimic Intent, Not just Trajectories(MINT). We achieve this via multi-scale frequency-space tokenization, which enforces a spectral decomposition of action chunk representation. We learn action tokens with a multi-scale coarse-to-fine structure, and force the coarsest token to capture low-frequency global structure and finer tokens to encode high-frequency details. This yields an abstract Intent token that facilitates planning and transfer, and multi-scale Execution tokens that enable precise adaptation to environmental dynamics. Building on this hierarchy, our policy generates trajectories through next-scale autoregression, performing progressive intent-to-execution reasoning, thus boosting learning efficiency and generalization. Crucially, this disentanglement enables one-shot transfer of skills, by simply injecting the Intent token from a demonstration into the autoregressive generation process. Experiments on several manipulation benchmarks and on a real robot demonstrate state-of-the-art success rates, superior inference efficiency, robust generalization against disturbances, and effective one-shot transfer.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。