arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.04810cs.CV
收藏

EMMA: 一种高效的多模态理解、生成与编辑统一架构

EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture

Huawei Inc.(华为公司)

Xin He, Longhui Wei, Jianbo Ouyang, Minghui Liao, Lingxi Xie, Qi Tian

AI总结:

EMMA提出了一种高效的统一多模态架构,通过高效自编码器、通道级拼接、共享解耦网络和专家混合机制,在效率和性能上超越现有方法。

摘要:

我们提出了EMMA,一种高效的统一架构,用于多模态理解、生成和编辑。具体而言,EMMA主要由四个部分组成:1)一个具有32倍压缩比的高效自编码器,显著减少了生成所需标记的数量。这还通过将相同的压缩比应用于图像来确保在理解和生成任务之间保持训练平衡。2)在视觉理解和生成标记之间采用通道级拼接而非标记级拼接,进一步减少统一架构中的视觉标记。3)一个共享且解耦的网络,使任务之间能够相互提升,同时满足任务特定的建模需求。4)在视觉理解编码器中采用专家混合机制,通过少量参数的增加显著提高了感知能力。大量实验表明,EMMA-4B在效率和性能上均显著优于最先进的统一多模态方法(如BAGEL-7B),同时在多模态理解和生成专家(如Qwen3-VL和Qwen-Image)中也取得了具有竞争力的结果。我们相信EMMA为未来统一多模态架构的发展奠定了坚实基础。

英文摘要:

We propose EMMA, an efficient and unified architecture for multimodal understanding, generation and editing. Specifically, EMMA primarily consists of 1) An efficient autoencoder with a 32x compression ratio, which significantly reduces the number of tokens required for generation. This also ensures the training balance between understanding and generation tasks by applying the same compression ratio to images. 2) Channel-wise concatenation instead of token-wise concatenation among visual understanding and generation tokens, which further reduces the visual tokens in unified architectures. 3) A shared-and-decoupled network that enables mutual improvements across tasks while meeting the task-specific modeling requirements. 4) A mixture-of-experts mechanism adopted for visual understanding encoder, which substantially improves perceptual capabilities with a few parameters increase. Extensive experiments have shown that EMMA-4B can significantly outperform state-of-the-art unified multimodal approaches (e.g., BAGEL-7B) in both efficiency and performance, while also achieving competitive results compared to recent multimodal understanding and generation experts (e.g., Qwen3-VL and Qwen-Image). We believe that EMMA lays a solid foundation for the future development of unified multimodal architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑