arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.02629cs.CV
收藏

VidGen-1M:一个用于文本到视频生成的大规模数据集

VidGen-1M: A Large-Scale Dataset for Text-to-video Generation

Shanghai Academy of AI for Science(上海人工智能科学研究院);Fudan University(复旦大学)

Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, Hao Li

AI总结:

针对现有文本到视频训练数据质量低、分布不均等问题,本文提出VidGen-1M大规模数据集,采用从粗到细的筛选策略保证视频质量与字幕时间一致性,实验表明其训练效果优于其他模型。

摘要:

视频-文本对的质量从根本上决定了文本到视频模型的上限。目前,用于训练这些模型的数据集存在显著缺陷,包括时间一致性低、字幕质量差、视频质量不达标以及数据分布不均衡。主流的视频筛选流程依赖图像模型进行标注和基于人工规则的筛选,导致计算负担高,并遗留不干净的数据。因此,文本到视频模型缺乏合适的训练数据集。为解决这一问题,我们提出了VidGen-1M,一个用于文本到视频模型的优质训练数据集。该数据集通过从粗到细的筛选策略生成,保证了高质量的视频和具有出色时间一致性的详细字幕。当用于训练视频生成模型时,该数据集带来了超越其他模型所获结果的实验结果。

英文摘要:

The quality of video-text pairs fundamentally determines the upper bound of text-to-video models. Currently, the datasets used for training these models suffer from significant shortcomings, including low temporal consistency, poor-quality captions, substandard video quality, and imbalanced data distribution. The prevailing video curation process, which depends on image models for tagging and manual rule-based curation, leads to a high computational load and leaves behind unclean data. As a result, there is a lack of appropriate training datasets for text-to-video models. To address this problem, we present VidGen-1M, a superior training dataset for text-to-video models. Produced through a coarse-to-fine curation strategy, this dataset guarantees high-quality videos and detailed captions with excellent temporal consistency. When used to train the video generation model, this dataset has led to experimental results that surpass those obtained with other models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑