HP-Edit：一种用于图像编辑的人类偏好后训练框架
HP-Edit: A Human-Preference Post-Training Framework for Image Editing
AI总结：
本文提出HP-Edit框架和RealPref-50K数据集，通过少量人类偏好评分数据和预训练视觉大语言模型开发自动评估器，提升图像编辑模型对人类偏好的契合度。
摘要：
常见的图像编辑任务通常采用强大的生成扩散模型作为现实内容编辑的主要范式。尽管强化学习（RL）方法如Diffusion-DPO和Flow-GRPO进一步提升了生成质量，但将强化学习从人类反馈（RLHF）应用于基于扩散的编辑仍鲜有研究，由于缺乏可扩展的人类偏好数据集和针对多样化编辑需求定制的框架。为填补这一空白，我们提出了HP-Edit，一种用于人类偏好对齐的后训练框架，并引入了RealPref-50K，一个涵盖八个常见任务、平衡常见物体编辑的现实世界数据集。具体而言，HP-Edit利用少量人类偏好评分数据和预训练的视觉大语言模型（VLM）开发HP-Scorer——一个自动的人类偏好对齐评估器。我们随后利用HP-Scorer既高效构建可扩展的偏好数据集，又作为后训练编辑模型的奖励函数。我们还介绍了RealPref-Bench，一个评估现实世界编辑性能的基准。大量实验表明，我们的方法显著增强了如Qwen-Image-Edit-2509等模型，使它们的输出更贴近人类偏好。
英文摘要：
Common image editing tasks typically adopt powerful generative diffusion models as the leading paradigm for real-world content editing. Meanwhile, although reinforcement learning (RL) methods such as Diffusion-DPO and Flow-GRPO have further improved generation quality, efficiently applying Reinforcement Learning from Human Feedback (RLHF) to diffusion-based editing remains largely unexplored, due to a lack of scalable human-preference datasets and frameworks tailored to diverse editing needs. To fill this gap, we propose HP-Edit, a post-training framework for Human Preference-aligned Editing, and introduce RealPref-50K, a real-world dataset across eight common tasks and balancing common object editing. Specifically, HP-Edit leverages a small amount of human-preference scoring data and a pretrained visual large language model (VLM) to develop HP-Scorer--an automatic, human preference-aligned evaluator. We then use HP-Scorer both to efficiently build a scalable preference dataset and to serve as the reward function for post-training the editing model. We also introduce RealPref-Bench, a benchmark for evaluating real-world editing performance. Extensive experiments demonstrate that our approach significantly enhances models such as Qwen-Image-Edit-2509, aligning their outputs more closely with human preference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
