Aligning text-to-image diffusion model (T2I) with preference has been gaining increasing research attention. While prior works exist on directly optimizing T2I by preference data, these methods are developed under the bandit assumption of a latent reward on the entire diffusion reverse chain, while ignoring the sequential nature of the generation process. This may harm the efficacy and efficiency of preference alignment. In this paper, we take on a finer dense reward perspective and derive a tractable alignment objective that emphasizes the initial steps of the T2I reverse chain. In particular, we introduce temporal discounting into DPO-style explicit-reward-free objectives, to break the temporal symmetry therein and suit the T2I generation hierarchy. In experiments on single and multiple prompt generation, our method is competitive with strong relevant baselines, both quantitatively and qualitatively. Further investigations are conducted to illustrate the insight of our approach.
Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；Qing Yuan Research Institute, Shanghai Jiao Tong University（上海交通大学清源研究院）；Centre for Perceptual and Interactive Intelligence (CPII)（感知与交互智能中心（CPII））；Shanghai AI Laboratory（上海人工智能实验室）
Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation
City University of Hong Kong（香港城市大学）；ShanghaiTech University（上海科技大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）；The Ohio State University（俄亥俄州立大学）；University of Glasgow（格拉斯哥大学）；Columbia University（哥伦比亚大学）；University of Arizona（亚利桑那大学）；GMI Cloud；National University of Singapore（新加坡国立大学）；DeciLix Lab
FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；USC-SJTU Institute of Cultural and Creative Industry, Shanghai Jiao Tong University（上海交通大学 USC-文化创意产业学院）；Institut Universitaire de France (IUF)（法国大学研究院）；University of Nantes（南特大学）
Fusion in Your Way: Aligning Image Fusion with Heterogeneous Demands via Direct Preference Optimization
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education（社会计算与认知智能重点实验室（大连理工大学），教育部）；Institute of Image Processing and Understanding, North Minzu University（北华大学图像处理与理解研究所）
Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift
National University of Singapore（新加坡国立大学）；A*STAR Institute of Advanced Intelligence and Computing, Singapore（新加坡科技研究局先进智能与计算研究所）；South China University of Technology（华南理工大学）；University of Science and Technology of China（中国科学技术大学）；Google DeepMind（谷歌DeepMind）
Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
