arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2505.02471cs.CV
收藏

Ming-Lite-Uni:面向自然多模态交互的统一架构研究进展

Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction

Inclusion AI Ant Group(Inclusion AI蚂蚁集团)
Inclusion AI, Biao Gong, Cheng Zou, Dandan Zheng, Hu Yu, Jingdong Chen, Jianxin Sun, Junbo Zhao, Jun Zhou, Kaixiang Ji, Lixiang Ru, Libin Wang, Qingpei Guo, Rui… 展开作者

Inclusion AI, Biao Gong, Cheng Zou, Dandan Zheng, Hu Yu, Jingdong Chen, Jianxin Sun, Junbo Zhao, Jun Zhou, Kaixiang Ji, Lixiang Ru, Libin Wang, Qingpei Guo, Rui Liu, Weilong Chai, Xinyu Xiao, Ziyuan Huang

AI总结:

本研究推出开源多模态框架Ming-Lite-Uni,通过新型统一视觉生成器、多尺度可学习token与对齐策略,让原生多模态自回归模型兼具文生图与指令图像编辑能力,性能与交互流畅性优异,契合多模态AI发展方向,助力AGI探索。

摘要:

我们推出Ming-Lite-Uni,这是一个开源多模态框架,具备全新设计的统一视觉生成器以及为统一视觉与语言定制的原生多模态自回归模型。具体而言,该项目提供了集成化MetaQueries与M2-omni框架的开源实现,同时引入了新颖的多尺度可学习token以及多尺度表征对齐策略。通过利用固定的MLLM(多模态大语言模型)与可学习扩散模型,Ming-Lite-Uni让原生多模态AR(自回归)模型能够执行文本生成图像任务以及基于指令的图像编辑任务,将其能力从纯视觉理解范畴向外拓展。我们的实验结果证明了Ming-Lite-Uni的优异性能,也展现出其交互过程出色的流畅性。所有代码与模型权重均已开源,以推动学界与业界的进一步探索。值得注意的是,这项工作与同期的多模态AI里程碑(例如2025年3月25日更新的具备原生图像生成能力的ChatGPT-4o)相契合,凸显了Ming-Lite-Uni这类统一模型在通往AGI(通用人工智能)道路上的广泛意义。Ming-Lite-Uni目前处于alpha阶段,很快将得到进一步优化。

英文摘要:

We introduce Ming-Lite-Uni, an open-source multimodal framework featuring a newly designed unified visual generator and a native multimodal autoregressive model tailored for unifying vision and language. Specifically, this project provides an open-source implementation of the integrated MetaQueries and M2-omni framework, while introducing the novel multi-scale learnable tokens and multi-scale representation alignment strategy. By leveraging a fixed MLLM and a learnable diffusion model, Ming-Lite-Uni enables native multimodal AR models to perform both text-to-image generation and instruction based image editing tasks, expanding their capabilities beyond pure visual understanding. Our experimental results demonstrate the strong performance of Ming-Lite-Uni and illustrate the impressive fluid nature of its interactive process. All code and model weights are open-sourced to foster further exploration within the community. Notably, this work aligns with concurrent multimodal AI milestones - such as ChatGPT-4o with native image generation updated in March 25, 2025 - underscoring the broader significance of unified models like Ming-Lite-Uni on the path toward AGI. Ming-Lite-Uni is in alpha stage and will soon be further refined.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑