arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.01985cs.CV
收藏

MT-EditFlow:基于流匹配的多轮图像编辑强化学习

MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching

Apple(苹果公司);University of California, Los Angeles(加州大学洛杉矶分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);Lambda, Inc(Lambda公司)

Jiahui Huang, Yasi Zhang, Tianyu Chen, Shu Wang, Jianwen Xie, Oscar Leong, Mingyuan Zhou, Nanzhu Wang, Ying Nian Wu

AI总结:

提出MT-EditFlow框架,通过流匹配强化学习优化多轮图像编辑的奖励信号,解决单轮编辑模型在多轮交互中的失败和误差传播问题,显著提升多轮编辑性能。

摘要:

近年来,基于指令的图像编辑取得了重大突破,模型现在能够处理现实世界中的编辑需求,满足日常用户的实用性要求。然而,主要为单轮编辑训练的编辑模型在多轮编辑中常常失败——在这种自然的交互设置中,用户基于模型自身之前的输出迭代地细化图像。这种失败源于“全有或全无”的要求,即单次失败会破坏整个序列,以及误差传播,即暴露偏差导致编辑误差累积。为了解决这些挑战,我们引入了MT-EditFlow,一个流匹配强化学习框架,旨在优化序列图像编辑的奖励信号。MT-EditFlow整合了多轮视角和多奖励公式,为基于GRPO和NFT的强化学习方法提供了统一的结构。我们通过研究有效的轮次级聚合评分策略、权衡奖励偏差与方差的VLM推理模式以及防止奖励破解的优势融合级别,系统地分析和优化了奖励信号。我们的发现表明,将聚合优势广播到整个编辑轨迹中,有效地弥合了局部规划与全局多轮任务成功之间的差距。大量实验表明,MT-EditFlow在多种基础模型上显著提升了性能。值得注意的是,它在FLUX.1-Kontext-dev上将第3轮整体性能提升了6.85分,超越了Qwen-Image-Edit等最先进的开源模型。通过保持高边际成功率和减少暴露偏差,MT-EditFlow为视觉内容创作中更可靠、更自然的人机协作奠定了基础。

英文摘要:

Recent breakthroughs in instruction-based image editing have captured significant attention, as models are now capable of handling real-world editing demands with the practicality required by everyday users. However, editing models trained primarily for single-turn edits often break down in multi-turn editing--the natural interactive setting where a user iteratively refines an image based on the model's own previous outputs. This failure stems from the all-or-nothing requirement, where a single failed turn compromises the entire sequence, and error propagation, where exposure bias leads to compounding editing errors. To address these challenges, we introduce MT-EditFlow, a flow-matching reinforcement learning framework designed to optimize reward signals for sequential image editing. MT-EditFlow integrates a multi-turn perspective with a multi-reward formulation to provide a unified structure applicable to both GRPO and NFT-based reinforcement learning methods. We systematically analyze and optimize the reward signal by investigating effective scoring strategies for turn-level aggregation, VLM reasoning modes to trade off reward bias and variance, and advantage fusion levels to prevent reward hacking. Our findings reveal that broadcasting the aggregated advantage across the entire editing trajectory effectively bridges the gap between local planning and global multi-turn task success. Extensive experiments demonstrate that MT-EditFlow significantly improves performance across diverse base models. Notably, it boosts FLUX.1-Kontext-dev by 6.85 points in turn-3 overall performance, surpassing state-of-the-art open-source models such as Qwen-Image-Edit. By maintaining high marginal success rates and reducing exposure bias, MT-EditFlow provides a foundation for more reliable and natural human-AI collaboration in visual content creation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑