arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01624cs.CV
收藏

PISCES: 基于最优传输对齐奖励的无标注文本到视频后训练

PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards

Microsoft(微软);Stony Brook University(石溪大学)

Minh-Quan Le, Gaurav Mittal, Cheng Zhao, David Gu, Dimitris Samaras, Mei Chen

AI总结:

提出PISCES算法,通过双重最优传输对齐奖励模块,无需人工标注即可提升文本生成视频的质量和语义对齐,在VBench上超越现有方法。

摘要:

文本到视频(T2V)生成旨在合成具有高视觉质量和时间一致性且与输入文本语义对齐的视频。基于奖励的后训练已成为提高生成视频质量和语义对齐的有前景方向。然而,现有方法要么依赖大规模人工偏好标注,要么操作来自预训练视觉语言模型的未对齐嵌入,导致可扩展性有限或监督次优。我们提出$\texttt{PISCES}$,一种无标注的后训练算法,通过新颖的双重最优传输(OT)对齐奖励模块解决这些限制。为了将奖励信号与人类判断对齐,$\texttt{PISCES}$使用OT在分布和离散标记级别桥接文本和视频嵌入,使奖励监督能够实现两个目标:(i)分布级OT对齐的质量奖励,捕捉整体视觉质量和时间连贯性;(ii)离散标记级OT对齐的语义奖励,强制文本和视频标记之间的语义、时空对应。据我们所知,$\texttt{PISCES}$是首个通过OT视角改进生成后训练中无标注奖励监督的方法。在短和长视频生成上的实验表明,$\texttt{PISCES}$在VBench的质量和语义分数上均优于基于标注和无标注的方法,人工偏好研究进一步验证了其有效性。我们展示了双重OT对齐奖励模块与多种优化范式兼容,包括直接反向传播和强化学习微调。项目页面:此https URL

英文摘要:

Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training has emerged as a promising direction to improve the quality and semantic alignment of generated videos. However, recent methods either rely on large-scale human preference annotations or operate on misaligned embeddings from pre-trained vision-language models, leading to limited scalability or suboptimal supervision. We present $\texttt{PISCES}$, an annotation-free post-training algorithm that addresses these limitations via a novel Dual Optimal Transport (OT)-aligned Rewards module. To align reward signals with human judgment, $\texttt{PISCES}$ uses OT to bridge text and video embeddings at both distributional and discrete token levels, enabling reward supervision to fulfill two objectives: (i) a Distributional OT-aligned Quality Reward that captures overall visual quality and temporal coherence; and (ii) a Discrete Token-level OT-aligned Semantic Reward that enforces semantic, spatio-temporal correspondence between text and video tokens. To our knowledge, $\texttt{PISCES}$ is the first to improve annotation-free reward supervision in generative post-training through the lens of OT. Experiments on both short- and long-video generation show that $\texttt{PISCES}$ outperforms both annotation-based and annotation-free methods on VBench across Quality and Semantic scores, with human preference studies further validating its effectiveness. We show that the Dual OT-aligned Rewards module is compatible with multiple optimization paradigms, including direct backpropagation and reinforcement learning fine-tuning. Project page: https://roar-ai.github.io/pisces

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑