VPO：通过提示优化对齐文本到视频生成模型
VPO: Aligning Text-to-Video Generation Models with Prompt Optimization
AI总结：
针对用户输入与训练数据描述存在差距导致视频生成质量不佳的问题，提出基于三大原则的两阶段VPO提示优化框架，显著提升视频生成的安全性、对齐度与质量，且泛化性强。
摘要：
视频生成模型在文本到视频任务中已取得显著进展。这类模型通常基于带有高度详尽、精心撰写描述的文本-视频对训练，而推理阶段真实世界的用户输入往往简洁、模糊或结构混乱。这种差距使得提示优化对于生成高质量视频至关重要。当前方法常依赖大语言模型（LLM）通过上下文学习优化提示，但存在若干局限：可能扭曲用户意图、遗漏关键细节或引入安全风险。此外，它们在优化提示时未考虑对最终视频质量的影响，可能导致结果未达最优。为解决这些问题，我们提出VPO，一个基于无害性、准确性、有用性三大核心原则优化提示的系统性框架。生成的提示能忠实保留用户意图，更重要的是提升了生成视频的安全性与质量。为实现这一目标，VPO采用两阶段优化方法：第一，基于安全与对齐原则构建并完善监督微调（SFT）数据集；第二，引入文本级和视频级反馈，通过偏好学习进一步优化SFT模型。大量实验表明，与基线方法相比，VPO显著提升了安全性、对齐度和视频质量，且在不同视频生成模型上展现出强泛化性。此外，我们证明VPO在视频生成模型上的表现优于RLHF方法，且可与RLHF方法结合，凸显了VPO在对齐视频生成模型方面的有效性。我们的代码和数据已在https://github.com/thu-coai/VPO 公开。
英文摘要：
Video generation models have achieved remarkable progress in text-to-video tasks. These models are typically trained on text-video pairs with highly detailed and carefully crafted descriptions, while real-world user inputs during inference are often concise, vague, or poorly structured. This gap makes prompt optimization crucial for generating high-quality videos. Current methods often rely on large language models (LLMs) to refine prompts through in-context learning, but suffer from several limitations: they may distort user intent, omit critical details, or introduce safety risks. Moreover, they optimize prompts without considering the impact on the final video quality, which can lead to suboptimal results. To address these issues, we introduce VPO, a principled framework that optimizes prompts based on three core principles: harmlessness, accuracy, and helpfulness. The generated prompts faithfully preserve user intents and, more importantly, enhance the safety and quality of generated videos. To achieve this, VPO employs a two-stage optimization approach. First, we construct and refine a supervised fine-tuning (SFT) dataset based on principles of safety and alignment. Second, we introduce both text-level and video-level feedback to further optimize the SFT model with preference learning. Our extensive experiments demonstrate that VPO significantly improves safety, alignment, and video quality compared to baseline methods. Moreover, VPO shows strong generalization across video generation models. Furthermore, we demonstrate that VPO could outperform and be combined with RLHF methods on video generation models, underscoring the effectiveness of VPO in aligning video generation models. Our code and data are publicly available at https://github.com/thu-coai/VPO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
