InternVL-U: 使统一多模态模型在理解、推理、生成和编辑方面更加普及
InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
AI总结:
InternVL-U通过轻量级统一多模态模型,在保持强大生成能力的同时,实现了在理解、推理、生成和编辑方面的高效平衡。
摘要:
统一多模态模型(UMMs)在整合理解、推理、生成和编辑方面面临保持强语义理解能力和获取强大生成能力之间的固有权衡。在本报告中,我们提出了InternVL-U,这是一个轻量级的40亿参数UMM,使其在统一框架内普及这些能力。受统一上下文建模和模态特定模块化设计与解耦视觉表示原则的指导,InternVL-U集成了最先进的多模态大语言模型(MLLM)与专门的MMDiT基于视觉生成头。为进一步弥合审美生成与高水平智能之间的差距,我们构建了一个全面的数据合成管道,针对高语义密度任务,如文本渲染和科学推理,在以推理为中心的范式下进行,利用链式思维(CoT)更好地对齐抽象用户意图与细粒度视觉生成细节。广泛的实验表明,InternVL-U在性能-效率平衡方面实现了优越的表现。尽管仅使用40亿参数,它在各种生成和编辑任务中 consistently 超过统一基准模型,如BAGEL(14B),同时保持强大的多模态理解和推理能力。
英文摘要:
Unified multimodal models (UMMs) that integrate understanding, reasoning, generation, and editing face inherent trade-offs between maintaining strong semantic comprehension and acquiring powerful generation capabilities. In this report, we present InternVL-U, a lightweight 4B-parameter UMM that democratizes these capabilities within a unified framework. Guided by the principles of unified contextual modeling and modality-specific modular design with decoupled visual representations, InternVL-U integrates a state-of-the-art Multimodal Large Language Model (MLLM) with a specialized MMDiT-based visual generation head. To further bridge the gap between aesthetic generation and high-level intelligence, we construct a comprehensive data synthesis pipeline targeting high-semantic-density tasks, such as text rendering and scientific reasoning, under a reasoning-centric paradigm that leverages Chain-of-Thought (CoT) to better align abstract user intent with fine-grained visual generation details. Extensive experiments demonstrate that InternVL-U achieves a superior performance - efficiency balance. Despite using only 4B parameters, it consistently outperforms unified baseline models with over 3x larger scales such as BAGEL (14B) on various generation and editing tasks, while retaining strong multimodal understanding and reasoning capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。