VidCapBench：面向可控文本生成视频的视频字幕综合基准
VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation
AI总结：
本研究提出专为可控文本生成视频设计的视频字幕评估基准VidCapBench，通过结合专家模型标注与人工优化的流水线构建多维度评估体系，验证显示其评估得分与T2V生成质量显著正相关，可为T2V模型训练提供指导。
摘要：
可控文本生成视频（T2V）模型的训练高度依赖视频与字幕之间的对齐性，但现有研究很少将视频字幕评估与T2V生成评估联系起来。本文提出VidCapBench，这是一种专门为T2V生成设计的视频字幕评估方案，不依赖任何特定的字幕格式。VidCapBench采用一套数据标注流水线，结合专家模型标注与人工优化，为每段收集到的视频关联涵盖视频美学、内容、运动和物理规律的关键信息。随后VidCapBench将这些关键信息属性划分为可自动评估和可人工评估的子集，同时满足敏捷开发的快速评估需求和全面验证的准确性要求。通过评估多个最先进的字幕模型，我们证明VidCapBench相比现有视频字幕评估方法具备更优的稳定性和全面性。使用现成T2V模型进行的验证显示，VidCapBench得分与T2V质量评估指标之间存在显著正相关，表明VidCapBench能为T2V模型训练提供有价值的指导。该项目可在https://github.com/VidCapBench/VidCapBench获取。
英文摘要：
The training of controllable text-to-video (T2V) models relies heavily on the alignment between videos and captions, yet little existing research connects video caption evaluation with T2V generation assessment. This paper introduces VidCapBench, a video caption evaluation scheme specifically designed for T2V generation, agnostic to any particular caption format. VidCapBench employs a data annotation pipeline, combining expert model labeling and human refinement, to associate each collected video with key information spanning video aesthetics, content, motion, and physical laws. VidCapBench then partitions these key information attributes into automatically assessable and manually assessable subsets, catering to both the rapid evaluation needs of agile development and the accuracy requirements of thorough validation. By evaluating numerous state-of-the-art captioning models, we demonstrate the superior stability and comprehensiveness of VidCapBench compared to existing video captioning evaluation approaches. Verification with off-the-shelf T2V models reveals a significant positive correlation between scores on VidCapBench and the T2V quality evaluation metrics, indicating that VidCapBench can provide valuable guidance for training T2V models. The project is available at https://github.com/VidCapBench/VidCapBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
