arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.12490cs.CVcs.AIcs.LGcs.MM
收藏

InstructVideo:用人类反馈指导视频扩散模型

InstructVideo: Instructing Video Diffusion Models with Human Feedback

Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团);Tsinghua University(清华大学);Singapore University of Technology and Design(新加坡科技设计大学);Nanyang Technological University(南洋理工大学);University of Cambridge(剑桥大学)

Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng, Yining Pan, Yingya Zhang, Ziwei Liu, Samuel Albanie, Dong Ni

AI总结:

InstructVideo通过奖励微调将人类反馈融入视频扩散模型,采用部分DDIM推断和分段稀疏采样降低微调成本,并复用图像奖励模型提升生成视频质量,兼顾效率与泛化。

摘要:

扩散模型已成为视频生成的默认范式。然而,它们依赖于网络规模但质量参差不齐的数据,往往产生视觉上不吸引人且与文本提示不一致的结果。为解决此问题,我们提出InstructVideo,通过奖励微调用人类反馈指导文本到视频扩散模型。InstructVideo有两个关键组成部分:1)为缓解通过完整DDIM采样链生成所带来的奖励微调成本,我们将奖励微调重新构想为编辑。通过利用扩散过程破坏采样视频,InstructVideo仅需部分推断DDIM采样链,从而降低微调成本并提高微调效率。2)为弥补缺乏针对人类偏好的专用视频奖励模型,我们重新利用已建立的图像奖励模型,如HPSv2。为此,我们提出分段视频奖励(Segmental Video Reward),一种基于分段稀疏采样提供奖励信号的机制,以及时间衰减奖励(Temporally Attenuated Reward),一种在微调期间减轻时间建模退化的方法。大量定性和定量实验验证了在InstructVideo中使用图像奖励模型的实用性和有效性,显著提高了生成视频的视觉质量,同时不损害泛化能力。代码和模型将公开发布。

英文摘要:

Diffusion models have emerged as the de facto paradigm for video generation. However, their reliance on web-scale data of varied quality often yields results that are visually unappealing and misaligned with the textual prompts. To tackle this problem, we propose InstructVideo to instruct text-to-video diffusion models with human feedback by reward fine-tuning. InstructVideo has two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by generating through the full DDIM sampling chain, we recast reward fine-tuning as editing. By leveraging the diffusion process to corrupt a sampled video, InstructVideo requires only partial inference of the DDIM sampling chain, reducing fine-tuning cost while improving fine-tuning efficiency. 2) To mitigate the absence of a dedicated video reward model for human preferences, we repurpose established image reward models, e.g., HPSv2. To this end, we propose Segmental Video Reward, a mechanism to provide reward signals based on segmental sparse sampling, and Temporally Attenuated Reward, a method that mitigates temporal modeling degradation during fine-tuning. Extensive experiments, both qualitative and quantitative, validate the practicality and efficacy of using image reward models in InstructVideo, significantly enhancing the visual quality of generated videos without compromising generalization capabilities. Code and models will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑