VF-Eval：评估多模态LLM对AIGC视频生成反馈的能力
VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos
AI总结：
提出VF-Eval基准，通过四项任务评估MLLMs对AIGC视频的反馈能力，发现前沿模型表现有限，并验证对齐人类反馈可改进视频生成。
摘要：
MLLMs近期在视频问答领域被广泛研究。然而，现有大多数评估集中于自然视频，忽略了合成视频，如AI生成内容（AIGC）。同时，视频生成的一些工作依赖MLLMs评估生成视频的质量，但MLLMs在解释AIGC视频方面的能力仍 largely underexplored。为此，我们提出新基准VF-Eval，引入四项任务——连贯性验证、错误感知、错误类型检测和推理评估——以全面评估MLLMs在AIGC视频上的能力。我们在VF-Eval上评估13个前沿MLLMs，发现即使是表现最好的模型GPT-4.1，也难以在所有任务上持续取得良好表现。这凸显了我们基准的挑战性。此外，为研究VF-Eval在改进视频生成中的实际应用，我们开展实验RePrompt，表明使MLLMs更贴近人类反馈可有益于视频生成。
英文摘要：
MLLMs have been widely studied for video question answering recently. However, most existing assessments focus on natural videos, overlooking synthetic videos, such as AI-generated content (AIGC). Meanwhile, some works in video generation rely on MLLMs to evaluate the quality of generated videos, but the capabilities of MLLMs on interpreting AIGC videos remain largely underexplored. To address this, we propose a new benchmark, VF-Eval, which introduces four tasks-coherence validation, error awareness, error type detection, and reasoning evaluation-to comprehensively evaluate the abilities of MLLMs on AIGC videos. We evaluate 13 frontier MLLMs on VF-Eval and find that even the best-performing model, GPT-4.1, struggles to achieve consistently good performance across all tasks. This highlights the challenging nature of our benchmark. Additionally, to investigate the practical applications of VF-Eval in improving video generation, we conduct an experiment, RePrompt, demonstrating that aligning MLLMs more closely with human feedback can benefit video generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
