arXiv 2412.067822025-08-10cs.ROcs.CV

CARP：通过粗到细自回归预测进行视觉运动策略学习

CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Prediction

Westlake University（西湖大学）；Zhejiang University（浙江大学）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算先进创新中心）

Zhefei Gong, Pengxiang Ding, Shangke Lyu, Siteng Huang, Mingyang Sun, Wei Zhao, Zhaoxin Fan, Donglin Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出CARP，一种粗到细自回归视觉运动策略学习范式，通过动作自编码器与GPT风格transformer两阶段生成动作，在保持高效率的同时实现超越扩散模型的精度与10倍推理加速。

摘要：

在机器人视觉运动策略学习中，与传统自回归模型相比，基于扩散的模型在提高动作轨迹生成精度方面取得了显著成功。然而，由于多个去噪步骤和复杂约束带来的灵活性受限，它们存在效率低下的问题。在本文中，我们提出了粗到细自回归策略（CARP），这是一种用于视觉运动策略学习的新范式，它将自回归动作生成过程重新定义为一种粗到细的下一尺度方法。CARP将动作生成解耦为两个阶段：首先，动作自编码器学习整个动作序列的多尺度表示；然后，GPT风格的transformer通过粗到细的自回归过程细化序列预测。这种直观的方法能产生高精度且平滑的动作，其性能匹配甚至超越了基于扩散的策略，同时保持了与自回归策略相当的效率。我们在多种设置下进行了广泛评估，包括基于状态和基于图像的模拟基准上的单任务和多任务场景，以及真实世界任务。CARP实现了具有竞争力的成功率，提升高达10%，并且与最先进的策略相比，推理速度提高了10倍，为机器人任务中的动作生成建立了一个高性能、高效且灵活的范式。

英文摘要：

In robotic visuomotor policy learning, diffusion-based models have achieved significant success in improving the accuracy of action trajectory generation compared to traditional autoregressive models. However, they suffer from inefficiency due to multiple denoising steps and limited flexibility from complex constraints. In this paper, we introduce Coarse-to-Fine AutoRegressive Policy (CARP), a novel paradigm for visuomotor policy learning that redefines the autoregressive action generation process as a coarse-to-fine, next-scale approach. CARP decouples action generation into two stages: first, an action autoencoder learns multi-scale representations of the entire action sequence; then, a GPT-style transformer refines the sequence prediction through a coarse-to-fine autoregressive process. This straightforward and intuitive approach produces highly accurate and smooth actions, matching or even surpassing the performance of diffusion-based policies while maintaining efficiency on par with autoregressive policies. We conduct extensive evaluations across diverse settings, including single-task and multi-task scenarios on state-based and image-based simulation benchmarks, as well as real-world tasks. CARP achieves competitive success rates, with up to a 10% improvement, and delivers 10x faster inference compared to state-of-the-art policies, establishing a high-performance, efficient, and flexible paradigm for action generation in robotic tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。