MVP: 通过自监督掩码视频预测增强视频大型语言模型
MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction
AI总结：
MVP通过自监督掩码视频预测提升视频大语言模型的时间推理和因果理解能力。
摘要：
基于强化学习的视频大型语言模型（VideoLLMs）的后训练范式通过优化视觉-语义任务（如描述或视频问答）取得了显著成功。然而，尽管这些方法有效增强了感知能力，但它们主要针对整体内容理解，往往缺乏对内在时间一致性和帧间相关性的显式监督。这种倾向限制了模型捕捉复杂动态和细粒度视觉因果关系的能力。为了明确弥合这一差距，我们提出了一种新的后训练目标：掩码视频预测（MVP）。通过要求模型从一组具有挑战性的干扰因素中重建被掩码的连续片段，MVP迫使模型关注事件的序列逻辑和时间上下文。为了支持可扩展的训练，我们引入了一条可扩展的数据合成管道，能够将任意视频语料库转换为MVP训练样本，并进一步采用具有细粒度奖励函数的组相对策略优化（GRPO）来增强模型对视频上下文和时间属性的理解。全面的评估表明，MVP通过直接强化时间推理和因果理解来增强视频推理能力。
英文摘要：
Reinforcement learning based post-training paradigms for Video Large Language Models (VideoLLMs) have achieved significant success by optimizing for visual-semantic tasks such as captioning or VideoQA. However, while these approaches effectively enhance perception abilities, they primarily target holistic content understanding, often lacking explicit supervision for intrinsic temporal coherence and inter-frame correlations. This tendency limits the models' ability to capture intricate dynamics and fine-grained visual causality. To explicitly bridge this gap, we propose a novel post-training objective: Masked Video Prediction (MVP). By requiring the model to reconstruct a masked continuous segment from a set of challenging distractors, MVP forces the model to attend to the sequential logic and temporal context of events. To support scalable training, we introduce a scalable data synthesis pipeline capable of transforming arbitrary video corpora into MVP training samples, and further employ Group Relative Policy Optimization (GRPO) with a fine-grained reward function to enhance the model's understanding of video context and temporal properties. Comprehensive evaluations demonstrate that MVP enhances video reasoning capabilities by directly reinforcing temporal reasoning and causal understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
