arXiv 2606.111882026-06-09cs.CV

ARM: 一种具有统一离散表示的自回归大型多模态模型

ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations

Shanghai Key Lab of Intelligent Information Processing, Fudan University（复旦大学上海智能信息处理重点实验室）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；Youtu Lab, Tencent（腾讯优图实验室）；Meta AI；Shanghai AI Laboratory（上海人工智能实验室）

Junke Wang, Xiao Wang, Jiacheng Pan, Xuefeng Hu, Feng Li, Jingxiang Sun, Chaorui Deng, Zilong Chen, Yunpeng Chen, Kaibin Tian, Matthew Gwilliam, Hao Chen, Danhu… 展开作者

Junke Wang, Xiao Wang, Jiacheng Pan, Xuefeng Hu, Feng Li, Jingxiang Sun, Chaorui Deng, Zilong Chen, Yunpeng Chen, Kaibin Tian, Matthew Gwilliam, Hao Chen, Danhui Guan, Kun Xu, Weilin Huang, Zuxuan Wu, Haoqi Fan, Yu-Gang Jiang, Zhenheng Yang

备注：: technical report

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出ARM模型，通过离散语义视觉分词器将图像映射为紧凑token序列，结合自回归建模和强化学习，统一实现图像理解、生成和编辑，并提升任务性能与跨任务协同。

摘要：

本文介绍了ARM，一种基于离散表示的自回归模型，它在下一个词预测框架内统一了图像理解、生成和编辑。ARM建立在三个努力之上：首先，我们训练了一个离散语义视觉分词器，将图像映射为紧凑的token序列。我们的分词器通过多个目标进行监督，这些目标共同促进语义可辨别性、语言对齐和忠实重建，从而在共享潜在空间中支持多样化的任务。在此基础上，我们在大规模文本和图像token序列上训练了一个7B自回归模型，无缝地发展了视觉-语言感知和生成能力。最后，为了进一步改善文本到图像生成和指令引导编辑的偏好对齐行为，ARM应用强化学习（RL）来优化任务级目标，如视觉质量、指令遵循和编辑一致性。令人惊讶的是，结果表明RL不仅显著提高了目标任务的性能（例如，将WISE总体从0.50提升到0.56，GEdit-Bench-EN G_O从5.75提升到6.68），而且还诱导了文本到图像生成和编辑之间的跨任务协同。总的来说，这些发现凸显了自回归建模，当与强大的表示和偏好优化相结合时，作为多模态智能的可扩展基础。代码：此https URL。

英文摘要：

This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework. ARM is built on three efforts: first, we train a discrete semantic visual tokenizer that maps images into compact token sequences. Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment and faithful reconstruction, thereby supporting diverse tasks in a shared latent space. With this, we train a 7B autoregressive model over large-scale text and image token sequences, seamlessly developing vision-language perception and generation capabilities. Finally, to further improve preference-aligned behavior for text-to-image generation and instruction-guided editing, ARM applies reinforcement learning (RL) to optimize task-level objectives such as visual quality, instruction adherence, and edit consistency. Surprisingly, the results show that RL not only substantially improves performance on the target tasks (e.g., raising WISE overall from 0.50 to 0.56, GEdit-Bench-EN G_O from 5.75 to 6.68), but also induces cross-task synergy between text-to-image generation and editing. Collectively, these findings highlight autoregressive modeling, when paired with strong representations and preference optimization, as a scalable foundation for multimodal intelligence. Code: https://github.com/wdrink/ARM.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。