AIGVE-MACS:面向AI生成视频评估的统一多方面评论与评分模型
AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation
AI总结:
本文提出统一模型AIGVE-MACS,结合大规模基准AIGVE-BENCH 2、逐token加权损失和动态帧采样,同时输出评分与多方面评论,在AI生成视频评估中超越GPT-4o等基线,并驱动视频生成质量提升53.5%。
摘要:
AI生成视频模型的快速发展催生了对稳健且可解释评估框架的迫切需求。现有指标仅限于生成数值分数而缺乏解释性评论,导致可解释性低且与人类评估对齐度不足。为应对这些挑战,我们提出了AIGVE-MACS,一个统一的AI生成视频评估(AIGVE)模型,它不仅能在评估这些生成视频时提供数值分数,还能提供多方面语言评论反馈。我们方法的核心是AIGVE-BENCH 2,一个大规模基准,包含2,500个AI生成视频和22,500条人工标注的详细评论及数值分数,覆盖九个关键评估方面。利用AIGVE-BENCH 2,AIGVE-MACS结合了最新的视觉-语言模型,并采用新颖的逐token加权损失和动态帧采样策略,以更好地与人类评估者对齐。在监督和零样本基准上的全面实验表明,AIGVE-MACS在评分相关性和评论质量方面均达到了最先进的性能,显著优于包括GPT-4o和VideoScore在内的先前基线。此外,我们进一步展示了一个多智能体优化框架,其中来自AIGVE-MACS的反馈驱动视频生成的迭代改进,带来了53.5%的质量提升。这项工作为AI生成视频的全面、与人类对齐的评估建立了新范式。我们在https://huggingface.co/xiaoliux/AIGVE-MACS发布了AIGVE-BENCH 2和AIGVE-MACS。
英文摘要:
The rapid advancement of AI-generated video models has created a pressing need for robust and interpretable evaluation frameworks. Existing metrics are limited to producing numerical scores without explanatory comments, resulting in low interpretability and human evaluation alignment. To address those challenges, we introduce AIGVE-MACS, a unified model for AI-Generated Video Evaluation(AIGVE), which can provide not only numerical scores but also multi-aspect language comment feedback in evaluating these generated videos. Central to our approach is AIGVE-BENCH 2, a large-scale benchmark comprising 2,500 AI-generated videos and 22,500 human-annotated detailed comments and numerical scores across nine critical evaluation aspects. Leveraging AIGVE-BENCH 2, AIGVE-MACS incorporates recent Vision-Language Models with a novel token-wise weighted loss and a dynamic frame sampling strategy to better align with human evaluators. Comprehensive experiments across supervised and zero-shot benchmarks demonstrate that AIGVE-MACS achieves state-of-the-art performance in both scoring correlation and comment quality, significantly outperforming prior baselines including GPT-4o and VideoScore. In addition, we further showcase a multi-agent refinement framework where feedback from AIGVE-MACS drives iterative improvements in video generation, leading to 53.5% quality enhancement. This work establishes a new paradigm for comprehensive, human-aligned evaluation of AI-generated videos. We release the AIGVE-BENCH 2 and AIGVE-MACS at https://huggingface.co/xiaoliux/AIGVE-MACS.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。