GT-SVJ: 基于生成式Transformer的自监督视频评判器用于高效视频奖励建模
GT-SVJ: Generative-Transformer-Based Self-Supervised Video Judge For Efficient Video Reward Modeling
AI总结：
提出GT-SVJ，通过将视频生成模型重新用作能量模型，并利用对比目标训练，实现高效、时间感知的视频质量评估，在少量标注下达到SOTA。
摘要：
将视频生成模型与人类偏好对齐仍然具有挑战性：当前方法依赖视觉语言模型（VLM）进行奖励建模，但这些模型难以捕捉细微的时间动态。我们提出了一种根本不同的方法：将天生设计用于建模时间结构的视频生成模型重新用作奖励模型。我们提出了基于生成式Transformer的自监督视频评判器（GT-SVJ），这是一种新颖的评估模型，将最先进的视频生成模型转化为强大的时间感知奖励模型。我们的关键洞察是，生成模型可以重新表述为基于能量的模型（EBM），该模型为高质量视频分配低能量，为退化视频分配高能量，从而在通过对比目标训练时能够以惊人的精度区分视频质量。为了防止模型利用真实视频与生成视频之间的表面差异，我们通过受控的潜在空间扰动设计了具有挑战性的合成负样本：时间切片、特征交换和帧洗牌，这些模拟了真实但细微的视觉退化。这迫使模型学习有意义的时空特征，而不是琐碎的伪影。GT-SVJ在GenAI-Bench和MonteBench上仅使用30K人工标注就达到了最先进的性能：比现有的基于VLM的方法少6倍到65倍。
英文摘要：
Aligning video generative models with human preferences remains challenging: current approaches rely on Vision-Language Models (VLMs) for reward modeling, but these models struggle to capture subtle temporal dynamics. We propose a fundamentally different approach: repurposing video generative models, which are inherently designed to model temporal structure, as reward models. We present the Generative-Transformer-based Self-Supervised Video Judge (\modelname), a novel evaluation model that transforms state-of-the-art video generation models into powerful temporally-aware reward models. Our key insight is that generative models can be reformulated as energy-based models (EBMs) that assign low energy to high-quality videos and high energy to degraded ones, enabling them to discriminate video quality with remarkable precision when trained via contrastive objectives. To prevent the model from exploiting superficial differences between real and generated videos, we design challenging synthetic negative videos through controlled latent-space perturbations: temporal slicing, feature swapping, and frame shuffling, which simulate realistic but subtle visual degradations. This forces the model to learn meaningful spatiotemporal features rather than trivial artifacts. \modelname achieves state-of-the-art performance on GenAI-Bench and MonteBench using only 30K human-annotations: $6\times$ to $65\times$ fewer than existing VLM-based approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
