arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.05977cs.CVcs.LG
收藏

ImageReward:学习与评估文生图的人类偏好

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

Tsinghua University(清华大学);Zhipu AI(智谱AI);Beijing U. of Posts and Telecommunications(北京邮电大学)

Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, Yuxiao Dong

AI总结:

本文提出ImageReward奖励模型和ReFL调优算法,利用人类偏好反馈改进文生图模型,并通过自动和人类评估验证其有效性。

摘要:

我们提出了一种全面的解决方案,用于从人类偏好反馈中学习和改进文生图模型。首先,我们构建了ImageReward——首个通用型文生图人类偏好奖励模型——以有效编码人类偏好。其训练基于我们系统的标注流程,包括评分和排序,迄今已收集137k个专家比较。在人类评估中,ImageReward优于现有评分模型和指标,使其成为评估文生图合成的一个有前景的自动指标。在此基础上,我们提出了奖励反馈学习(ReFL),一种直接调优算法,用于针对评分器优化扩散模型。自动和人类评估均支持ReFL相对于比较方法的优势。所有代码和数据集均可在\url{https://github.com/THUDM/ImageReward}获取。

英文摘要:

We present a comprehensive solution to learn and improve text-to-image models from human preference feedback. To begin with, we build ImageReward -- the first general-purpose text-to-image human preference reward model -- to effectively encode human preferences. Its training is based on our systematic annotation pipeline including rating and ranking, which collects 137k expert comparisons to date. In human evaluation, ImageReward outperforms existing scoring models and metrics, making it a promising automatic metric for evaluating text-to-image synthesis. On top of it, we propose Reward Feedback Learning (ReFL), a direct tuning algorithm to optimize diffusion models against a scorer. Both automatic and human evaluation support ReFL's advantages over compared methods. All code and datasets are provided at \url{https://github.com/THUDM/ImageReward}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑