Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation
School of Vehicle and Mobility & College of AI, Tsinghua University(车辆与移动学院及人工智能学院,清华大学);Berkeley AI Research (BAIR), UC Berkeley(伯克利人工智能研究(BAIR),加州大学伯克利分校);The University of Hong Kong(香港大学)
Learning expressive and efficient policy functions is a promising direction in reinforcement learning (RL). While flow-based policies have recently proven effective in modeling complex action distributions with a fast deterministic sampling process, they still face a trade-off between expressiveness and computational burden, which is typically controlled by the number of flow steps. In this work, we propose mean velocity policy (MVP), a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation. To ensure its high expressiveness, an instantaneous velocity constraint (IVC) is introduced on the mean velocity field during training. We theoretically prove that this design explicitly serves as a crucial boundary condition, thereby improving learning accuracy and enhancing policy expressiveness. Empirically, our MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench. It also delivers substantial improvements in training and inference speed over existing flow-based policy baselines.
Max-Entropy Reinforcement Learning with Flow Matching and A Case Study on LQR
School of Engineering and Applied Sciences, Harvard University(哈佛大学工程与应用科学学院);Harvard University(哈佛大学);College of Computing, Georgia Institute of Technology(佐治亚理工学院计算机学院)
Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning
Department of Computing Science, University of Alberta, Edmonton, AB, Canada(阿尔伯塔大学计算机科学系);Alberta Machine Intelligence Institute(阿尔伯塔机器智能研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席);Khoury College of Computer Sciences, Northeastern University, Boston, MA, USA(东北大学计算机科学学院)
MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference
Data Science & Artificial Intelligence Research Institute, China Unicom(中国unicom数据科学与人工智能研究 institute);Unicom Data Intelligence, China Unicom(中国unicom数据智能);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)