ID-Crafter：基于视觉语言模型的在线强化学习多主体视频生成框架
ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generation
AI总结：
本文提出ID-Crafter框架，通过层次化注意力机制、预训练视觉语言模型和在线强化学习，提升多主体视频生成中身份保持与语义一致性。
摘要：
在高保真视频合成领域取得显著进展，但现有方法在整合多主体身份信息方面效果有限，导致语义冲突和性能不足。为解决此问题，本文引入ID-Crafter框架，通过三级身份保持注意力机制、预训练视觉语言模型模块和在线强化学习阶段，实现更优的身份保持和语义一致性。此外，本文构建了新的数据集以促进鲁棒训练与评估。实验表明，ID-Crafter在多主体视频生成基准上取得新状态-of-the-art性能，尤其在身份保持、时间一致性及视频质量方面表现突出。项目页面：https://angericky.github.io/ID-Crafter
英文摘要：
Significant progress has been achieved in high-fidelity video synthesis, yet current paradigms often fall short in effectively integrating identity information from multiple subjects. This leads to semantic conflicts and suboptimal performance in preserving identities and interactions, limiting controllability and applicability. To tackle this issue, we introduce ID-Crafter, a framework for multi-subject video generation that achieves superior identity preservation and semantic coherence. ID-Crafter integrates three key components: (i) a hierarchical identity-preserving attention mechanism that progressively aggregates features at intra-subject, inter-subject, and cross-modal levels; (ii) a semantic understanding module powered by a pretrained Vision-Language Model (VLM) to provide fine-grained guidance and capture complex inter-subject relationships; and (iii) an online reinforcement learning phase to further refine the model for critical concepts. Furthermore, we construct a new dataset to facilitate robust training and evaluation. Extensive experiments demonstrate that ID-Crafter establishes new state-of-the-art performance on multi-subject video generation benchmarks, excelling in identity preservation, temporal consistency, and overall video quality. Project page: https://angericky.github.io/ID-Crafter
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
