发表机构
Tencent Hunyuan; The Hong Kong University of Science and Technology（腾讯混元; 香港科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究将强化学习应用于MeanFlow生成器的问题，核心方法是引入MeanFlowNFT，受MeanFlow恒等式启发构建预测器并应用DiffusionNFT目标，主要贡献是改进基线，在多数指标上超越现有方法，少步采样时能超越多步强化学习调整的扩散模型。
AI 中文摘要
MeanFlow生成器通过预测时间间隔内的平均速度实现快速少步采样，强化学习已成为使扩散和流模型符合人类偏好及特定任务目标的有力方式。DiffusionNFT提供了一个高效的前向过程强化学习框架，但将此类强化学习方法应用于MeanFlow的研究仍不足。为此引入MeanFlowNFT，受连接平均速度和瞬时速度的MeanFlow恒等式启发构建诱导瞬时速度预测器，应用DiffusionNFT目标进行奖励优化，同时保留基于平均速度的采样方式。实验表明MeanFlowNFT持续改进基线，在多数指标上优于现有方法。
英文摘要
MeanFlow generators achieve fast few-step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task-specific objectives. In particular, DiffusionNFT offers an efficient forward-process RL framework that does not require reverse-process trajectories or likelihood estimation. However, applying such RL methods to MeanFlow remains underexplored. DiffusionNFT optimizes instantaneous velocities, whereas MeanFlow samples with average velocities. To bridge this gap, we introduce MeanFlowNFT. Inspired by the MeanFlow identity, which bridges average and instantaneous velocities, we construct an induced instantaneous-velocity predictor. We apply the DiffusionNFT objective to this predictor, making reward optimization well-defined for MeanFlow. Sampling remains based on the average velocity, preserving MeanFlow's fast few-step generation. We further prove that MeanFlowNFT inherits DiffusionNFT's strict policy-improvement guarantee. Experiments on image and video generation show that MeanFlowNFT consistently improves baselines. Moreover, it outperforms prior state-of-the-art RL-tuned few-step generators on most metrics ($6$ of $8$ on SD3.5-M), and can even surpass multi-step RL-tuned diffusion while using only a few sampling steps. For instance, on Wan 2.1, $4$-step MeanFlowNFT reaches a VBench score of $84.33$, surpassing $50$-step LongCat-Video RL ($82.57$).
CommentsProject Page: https://harahan.github.io/meanflownft-project-page/, GitHub: https://github.com/Harahan/MeanFlowNFT, Hugging Face: https://huggingface.co/Harahan/MeanFlowNFT, Demo: https://huggingface.co/spaces/Harahan/meanflownft-fewstep-generation
