Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans. Although recent work attempts to learn these preferences via human annotated images, they reduce the rich tapestry of human preference to a single overall score. However, the preference results vary when humans evaluate images with different aspects. Therefore, to learn the multi-dimensional human preferences, we propose the Multi-dimensional Preference Score (MPS), the first multi-dimensional preference scoring model for the evaluation of text-to-image models. The MPS introduces the preference condition module upon CLIP model to learn these diverse preferences. It is trained based on our Multi-dimensional Human Preference (MHP) Dataset, which comprises 918,315 human preference choices across four dimensions (i.e., aesthetics, semantic alignment, detail quality and overall assessment) on 607,541 images. The images are generated by a wide range of latest text-to-image models. The MPS outperforms existing scoring methods across 3 datasets in 4 dimensions, enabling it a promising metric for evaluating and improving text-to-image generation.
HPSv3: Towards Wide-Spectrum Human Preference Score
Mizzen AI;CUHK MMLab(香港大学MMLab);King’s College London(伦敦国王学院);Shanghai Jiaotong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);CPII, InnoHK(创新香港科技促进会)
Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
The Chinese University of Hong Kong(香港中文大学);SenseTime Research(商汤科技研究院);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院);Centre for Perceptual and Interactive Intelligence (CPII)(感知与交互智能中心(CPII));Shanghai AI Laboratory(上海人工智能实验室)
Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
The Chinese University of Hong Kong(香港中文大学);SenseTime Research(商汤科技研究院);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院);Centre for Perceptual and Interactive Intelligence(感知与交互智能研究中心)
Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation
City University of Hong Kong(香港城市大学);ShanghaiTech University(上海科技大学);University of California, Berkeley(加利福尼亚大学伯克利分校);The Ohio State University(俄亥俄州立大学);University of Glasgow(格拉斯哥大学);Columbia University(哥伦比亚大学);University of Arizona(亚利桑那大学);GMI Cloud;National University of Singapore(新加坡国立大学);DeciLix Lab
FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所);USC-SJTU Institute of Cultural and Creative Industry, Shanghai Jiao Tong University(上海交通大学 USC-文化创意产业学院);Institut Universitaire de France (IUF)(法国大学研究院);University of Nantes(南特大学)