HIVE：利用人类反馈进行指令式视觉编辑
HIVE: Harnessing Human Feedback for Instructional Visual Editing
AI总结：
本文提出HIVE框架，通过收集人类反馈学习奖励函数并微调扩散模型，同时构建百万级新数据集，显著提升了指令式图像编辑的性能。
摘要：
已有研究表明，引入人类反馈对于使大型语言模型生成的文本符合人类偏好至关重要。我们假设，最先进的指令式图像编辑模型（即根据输入图像和编辑指令生成输出的模型）同样能从人类反馈中受益，因为其输出可能无法遵循用户的正确指令和偏好。本文提出了一个利用人类反馈进行指令式视觉编辑的新框架（HIVE）。具体而言，我们收集针对编辑图像的人类反馈，并学习一个奖励函数来捕捉潜在的用户偏好。随后，我们引入可扩展的扩散模型微调方法，该方法能基于估计的奖励融入人类偏好。此外，为缓解数据局限性带来的偏差，我们贡献了一个新的1M训练数据集、一个用于奖励学习的3.6K奖励数据集，以及一个1K评估数据集，以提升指令式图像编辑的性能。我们进行了大量定量和定性的实证实验，结果表明HIVE在很大程度上优于以往最先进的指令式图像编辑方法。
英文摘要：
Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences. We hypothesize that state-of-the-art instructional image editing models, where outputs are generated based on an input image and an editing instruction, could similarly benefit from human feedback, as their outputs may not adhere to the correct instructions and preferences of users. In this paper, we present a novel framework to harness human feedback for instructional visual editing (HIVE). Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward. Besides, to mitigate the bias brought by the limitation of data, we contribute a new 1M training dataset, a 3.6K reward dataset for rewards learning, and a 1K evaluation dataset to boost the performance of instructional image editing. We conduct extensive empirical experiments quantitatively and qualitatively, showing that HIVE is favored over previous state-of-the-art instructional image editing approaches by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
