Qwen-Image-2.0-RL 技术报告
Qwen-Image-2.0-RL Technical Report
AI总结：
提出基于强化学习与在线蒸馏的后训练流程，通过复合奖励模型和GRPO框架提升扩散模型的视觉质量与指令遵循能力，在多个基准上取得显著提升。
摘要：
我们提出了Qwen-Image-2.0-RL，一个后训练流程，应用从人类反馈中强化学习（RLHF）和在线策略蒸馏（OPD）来提升Qwen-Image-2.0扩散模型的视觉质量和指令遵循能力。为了提供可靠的奖励信号，我们通过微调视觉语言模型，采用逐点评分范式和思维链推理构建了任务特定的复合奖励模型。对于文本到图像生成，奖励模型涵盖对齐、美学和人像保真度维度。对于图像编辑任务，奖励系统处理指令遵循准确性和人脸身份保持。基于此奖励系统，我们开发了一个可扩展的基于GRPO的RL训练框架，结合混合无分类器引导（CFG）策略以保留预训练知识，通过组内奖励范围过滤进行提示整理，以及按类别奖励权重校准。为了合并T2I和编辑的任务特定RL策略，我们提出在线策略蒸馏作为最终训练阶段，通过轨迹级速度匹配将多个教师模型整合到单个学生模型中。广泛评估表明，Qwen-Image-2.0-RL在Qwen-Image-Bench上获得57.84总分（比基础模型高2.61），在文本到图像竞技场中Elo评级为1193（+78），在图像编辑竞技场中为1349（+93），在美学质量、提示遵循和编辑准确性方面均表现出持续提升。
英文摘要：
We present Qwen-Image-2.0-RL, a post-training pipeline that applies reinforcement learning from human feedback (RLHF) and on-policy distillation (OPD) to improve both the visual quality and instruction-following capability of the Qwen-Image-2.0 diffusion model. To provide reliable reward signals, we construct task-specific composite reward models by fine-tuning vision-language models with a pointwise scoring paradigm and chain-of-thought reasoning. For text-to-image generation, the reward models cover alignment, aesthetics, and portrait fidelity dimensions. For image editing tasks, the reward system addresses instruction-following accuracy and face identity preservation. Building on this reward system, we develop a scalable GRPO-based RL training framework, incorporating a hybrid classifier-free guidance (CFG) strategy to preserve pre-trained knowledge, prompt curation via intra-group reward range filtering, and per-category reward weight calibration. To merge the task-specialized RL policies for T2I and editing, we propose on-policy distillation as the final training stage, which consolidates multiple teachers into a single student model through trajectory-level velocity matching. Extensive evaluation shows that Qwen-Image-2.0-RL achieves 57.84 overall score on Qwen-Image-Bench (+2.61 over the base model), Elo ratings of 1193 in text-to-image arena (+78) and 1349 in image edit arena (+93), demonstrating consistent gains in aesthetic quality, prompt adherence, and editing accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
