arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13810cs.LGcs.AI
收藏

具有瞬时速度约束的一步动作生成均流策略

Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation

School of Vehicle and Mobility & College of AI, Tsinghua University(车辆与移动学院及人工智能学院,清华大学);Berkeley AI Research (BAIR), UC Berkeley(伯克利人工智能研究(BAIR),加州大学伯克利分校);The University of Hong Kong(香港大学)

Guojian Zhan, Letian Tao, Pengcheng Wang, Yixiao Wang, Yiheng Li, Yuxin Chen, Hongyang Li, Masayoshi Tomizuka, Shengbo Eben Li

AI总结:

本文提出均速度策略(MVP),通过引入瞬时速度约束来提升动作生成效率,实现高效的一步动作生成并在机器人操作任务中取得最佳性能。

摘要:

在强化学习(RL)中,学习具有表达性和效率的策略函数是一个有前景的方向。尽管基于流的策略最近在用快速确定性采样过程建模复杂动作分布方面证明了有效性,但它们仍然面临表达性和计算负担之间的权衡,这通常通过流步骤的数量来控制。在本工作中,我们提出了均速度策略(MVP),这是一种新的生成策略函数,通过建模均速度场来实现最快的一步动作生成。为了确保其高表达性,在训练期间在均速度场上引入了瞬时速度约束(IVC)。我们理论证明,这种设计明确地作为关键边界条件,从而提高学习精度并增强策略的表达性。实验表明,我们的MVP在Robomimic和OGBench中的多个具有挑战性的机器人操作任务中均取得了最先进的成功率。此外,它在现有基于流的策略基线的训练和推理速度上也带来了显著的改进。

英文摘要:

Learning expressive and efficient policy functions is a promising direction in reinforcement learning (RL). While flow-based policies have recently proven effective in modeling complex action distributions with a fast deterministic sampling process, they still face a trade-off between expressiveness and computational burden, which is typically controlled by the number of flow steps. In this work, we propose mean velocity policy (MVP), a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation. To ensure its high expressiveness, an instantaneous velocity constraint (IVC) is introduced on the mean velocity field during training. We theoretically prove that this design explicitly serves as a crucial boundary condition, thereby improving learning accuracy and enhancing policy expressiveness. Empirically, our MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench. It also delivers substantial improvements in training and inference speed over existing flow-based policy baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑