arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2501.08545cs.CV
收藏

T2VEval: 用于T2V生成视频的基准数据集和客观评估方法

T2VEval: Benchmark Dataset and Objective Evaluation Method for T2V-generated Videos

School of Information and Communication Engineering, Communication University of China, No.1 East Street, Dingfuzhuang, Chaoyang District, Beijing, China（信息与通信工程学院，中国传媒大学）；Beijing Academy of Artificial Intelligence, No.150 Chengfu Road, Haidian District, Beijing, China（北京人工智能研究院）

Zelu Qi, Ping Shi, Shuqi Wang, Chaoyang Zhang, Fei Zhao, Zefeng Ying, Da Pan, Xi Yang, Zheqi He, Teng Dai

备注：
This paper has been accepted by DISPLAYS
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

T2VEval提出了一种多分支融合方案，通过多维评估指标对T2V生成视频进行客观质量评估，提升视频生成模型的优化效果。

摘要：

最近在文本到视频（T2V）技术方面的进展，如Runway Gen-3、Pika、Sora和Kling等模型，显著扩大了该技术的应用范围和普及度。这一进展催生了对准确质量评估指标日益增长的需求，以评估T2V生成视频的感知质量并优化视频生成模型。然而，评估文本到视频输出的质量仍然具有挑战性，因为存在高度复杂的失真，如不自然的动作和违背人类认知的现象。为了解决这些挑战，我们构建了T2VEval-Bench，一个用于文本到视频质量评估的多维基准数据集，包含148个文本提示和1,783个由13个T2V模型生成的视频。为了确保全面评估，我们在主观实验中对每个视频在四个维度上进行评分，即总体印象、文本-视频一致性、真实性和技术质量。基于T2VEval-Bench，我们开发了T2VEval，一种用于T2V质量评估的多分支融合方案。T2VEval在三个分支上评估视频：文本-视频一致性、真实性和技术质量。使用基于注意力的融合模块，T2VEval有效地整合了每个分支的特征，并借助大语言模型预测评分。此外，我们实施了一种分而治之的训练策略，使每个分支能够学习特定的知识，同时保持与其他分支的协同作用。实验结果表明，T2VEval在多个指标上均达到最先进的性能。

英文摘要：

Recent advances in text-to-video (T2V) technology, as demonstrated by models such as Runway Gen-3, Pika, Sora, and Kling, have significantly broadened the applicability and popularity of the technology. This progress has created a growing demand for accurate quality assessment metrics to evaluate the perceptual quality of T2V-generated videos and optimize video generation models. However, assessing the quality of text-to-video outputs remain challenging due to the presence of highly complex distortions, such as unnatural actions and phenomena that defy human cognition. To address these challenges, we constructed T2VEval-Bench, a multi-dimensional benchmark dataset for text-to-video quality evaluation, which contains 148 textual prompts and 1,783 videos generated by 13 T2V models. To ensure a comprehensive evaluation, we scored each video on four dimensions in the subjective experiment, which are overall impression, text-video consistency, realness, and technical quality. Based on T2VEval-Bench, we developed T2VEval, a multi-branch fusion scheme for T2V quality evaluation. T2VEval assesses videos across three branches: text-video consistency, realness, and technical quality. Using an attention-based fusion module, T2VEval effectively integrates features from each branch and predicts scores with the aid of a large language model. Additionally, we implemented a divide-and-conquer training strategy, enabling each branch to learn targeted knowledge while maintaining synergy with the others. Experimental results demonstrate that T2VEval achieves state-of-the-art performance across multiple metrics.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑