HiFlow:通过流匹配实现无标记的分层自回归策略学习
HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching
AI总结:
HiFlow通过流匹配方法,无需标记直接在连续动作上进行分层自回归策略学习,有效提升了visuomotor策略学习的性能。
摘要:
粗到细的自回归建模最近在visuomotor策略学习中显示出强大前景,结合了自回归方法的推理效率与基于扩散策略的全局轨迹一致性。然而,现有方法依赖于离散动作标记器,将连续动作序列映射到代码本索引,这种设计继承自图像生成领域,需要学习压缩以处理高维像素数据。我们发现机器人动作本质上是低维连续向量,此类标记引入了不必要的量化误差和多阶段训练流程。在本文中,我们提出了分层流策略(HiFlow),一种无标记的粗到细自回归策略,直接在原始连续动作上操作。HiFlow通过简单的时序池化从每个动作块构建多尺度连续动作目标。具体来说,它通过平均连续动作窗口生成粗略摘要,并在更细的时间分辨率上进行细化。整个模型在单阶段内端到端训练,消除了对单独标记器的需求。在MimicGen、RoboTwin 2.0和现实环境中的实验表明,HiFlow在现有方法包括基于扩散和标记的自回归策略上表现一致。
英文摘要:
Coarse-to-fine autoregressive modeling has recently shown strong promise for visuomotor policy learning, combining the inference efficiency of autoregressive methods with the global trajectory coherence of diffusion-based policies. However, existing approaches rely on discrete action tokenizers that map continuous action sequences to codebook indices, a design inherited from image generation where learned compression is necessary for high-dimensional pixel data. We observe that robot actions are inherently low-dimensional continuous vectors, for which such tokenization introduces unnecessary quantization error and a multi-stage training pipeline. In this work, we propose Hierarchical Flow Policy (HiFlow), a tokenization-free coarse-to-fine autoregressive policy that operates directly on raw continuous actions. HiFlow constructs multi-scale continuous action targets from each action chunk via simple temporal pooling. Specifically, it averages contiguous action windows to produce coarse summaries that are refined at finer temporal resolutions. The entire model is trained end-to-end in a single stage, eliminating the need for a separate tokenizer. Experiments on MimicGen, RoboTwin 2.0, and real-world environments demonstrate that HiFlow consistently outperforms existing methods including diffusion-based and tokenization-based autoregressive policies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。