开放域定制视频生成的综合生态系统
A Comprehensive Ecosystem for Open-Domain Customized Video Generation
AI总结：
提出百万级数据集PexelsCustom-1M和参数高效框架CustoMDiT，仅用8%额外参数实现定制视频生成，并构建千类基准OpenCustom，开源整个生态系统。
摘要：
近期视频生成的进展展示了令人印象深刻的视觉合成能力。然而，开放域定制视频生成仍然受到缺乏大规模、带标注的数据集来捕捉多样化的身份特定属性的限制。为了解决这个问题，我们引入了PexelsCustom-1M，这是第一个公开可用的百万级身份保持视频生成数据集，包含跨越8000多个类别的一百万个精心策划的<身份，文本，视频>三元组。利用这一点，我们提出了CustoMDiT，一个参数高效的框架，将预训练的多模态扩散Transformer适配为定制视频生成器，仅增加8%的可学习参数。我们的方法超越了先前的最先进技术。然而，像DreamBooth这样的基准只覆盖了100个类别，对于现实应用来说是不够的。为了克服这一点，我们构建了OpenCustom，一个新的包含1000多个类别的基准，通过ImageNet和MS-COCO的跨数据集知识融合创建。大量实验证实了我们的数据集和模型的优势。我们将开源整个生态系统——包括数据集、流水线、基准和实现——以支持进一步的研究。
英文摘要：
Recent progress in video generation has shown impressive visual synthesis capabilities. However, open-domain customized video generation remains limited by the lack of large-scale, annotated datasets capturing diverse identity-specific attributes. To address this, we introduce PexelsCustom-1M, the first publicly available million-scale dataset for identity-preserving video generation, containing one million curated <identity, text, video> triplets across 8,000+ categories. Leveraging this, we propose CustoMDiT, a parameter-efficient framework that adapts a pretrained multimodal Diffusion Transformer into a customized video generator with only 8% additional learnable parameters. Our method surpasses prior state-of-the-art. However, benchmarks such as DreamBooth cover only 100 classes, which is insufficient for real-world applications. To overcome this, we construct OpenCustom, a new benchmark with 1,000+ categories, created via cross-dataset knowledge fusion from ImageNet and MS-COCO. Extensive experiments confirm the advantages of both our dataset and model. We will open-source the entire ecosystem--including dataset, pipeline, benchmark, and implementations--to support further research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
