Uni-ViGU：通过基于扩散的视频生成器实现视频生成与理解的统一
Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator
AI总结：
Uni-ViGU通过基于扩散的视频生成器统一视频生成与理解，引入统一流方法和模态驱动的MoE框架，实现多模态生成与理解的协同优化。
摘要：
统一的多模态模型整合视觉理解和生成面临根本性挑战：视觉生成的计算成本远高于理解，尤其在视频领域。这种不平衡促使我们颠覆传统范式：而非扩展以理解为中心的MLLMs以支持生成，我们提出Uni-ViGU框架，通过扩展视频生成器作为基础，实现视频生成与理解的统一。我们引入统一流方法，对视频进行连续流匹配，对文本进行离散流匹配，实现连贯的多模态生成。我们进一步提出一种模态驱动的MoE框架，通过在Transformer块中添加轻量级层来增强文本生成，同时保持生成先验。为将生成知识用于理解，我们设计了双向训练机制，包含两个阶段：知识回忆重建输入提示以利用学习的文本-视频对应关系，能力细化在详细描述上微调以建立判别性共享表示。实验表明，Uni-ViGU在视频生成和理解上均取得竞争性性能，验证了以生成为中心的架构作为统一多模态智能的可扩展路径。项目页面和代码：https://fr0zencrane.github.io/uni-vigu-page/.
英文摘要：
Unified multimodal models integrating visual understanding and generation face a fundamental challenge: visual generation incurs substantially higher computational costs than understanding, particularly for video. This imbalance motivates us to invert the conventional paradigm: rather than extending understanding-centric MLLMs to support generation, we propose Uni-ViGU, a framework that unifies video generation and understanding by extending a video generator as the foundation. We introduce a unified flow method that performs continuous flow matching for video and discrete flow matching for text within a single process, enabling coherent multimodal generation. We further propose a modality-driven MoE-based framework that augments Transformer blocks with lightweight layers for text generation while preserving generative priors. To repurpose generation knowledge for understanding, we design a bidirectional training mechanism with two stages: Knowledge Recall reconstructs input prompts to leverage learned text-video correspondences, while Capability Refinement fine-tunes on detailed captions to establish discriminative shared representations. Experiments demonstrate that Uni-ViGU achieves competitive performance on both video generation and understanding, validating generation-centric architectures as a scalable path toward unified multimodal intelligence. Project Page and Code: https://fr0zencrane.github.io/uni-vigu-page/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
