ANOLE：用于图文交错生成的开放、自回归、原生大型多模态模型
ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation
AI总结：
针对开源 LMM 依赖适配器、单模态生成及分离扩散模型等问题，本文基于 Chameleon 通过数据与参数高效微调构建原生自回归多模态模型 Anole，实现高质量图文交错生成并开源全部资源。
摘要：
此前的开源大型多模态模型（LMM）存在若干局限：（1）它们往往缺乏原生集成，需要使用适配器将视觉表示与预训练大语言模型（LLM）对齐；（2）许多模型仅限于单模态生成；（3）尽管部分模型支持多模态生成，但它们依赖独立的扩散模型进行视觉建模与生成。为缓解这些局限，我们提出 Anole，这是一个开放、自回归、原生的大型多模态模型，用于图文交错生成。我们基于 Meta AI 的 Chameleon 构建 Anole，并采用兼具数据效率与参数效率的创新微调策略。Anole 展现出高质量、连贯的多模态生成能力。我们已开源该模型、训练框架及指令微调数据。
英文摘要：
Previous open-source large multimodal models (LMMs) have faced several limitations: (1) they often lack native integration, requiring adapters to align visual representations with pre-trained large language models (LLMs); (2) many are restricted to single-modal generation; (3) while some support multimodal generation, they rely on separate diffusion models for visual modeling and generation. To mitigate these limitations, we present Anole, an open, autoregressive, native large multimodal model for interleaved image-text generation. We build Anole from Meta AI's Chameleon, adopting an innovative fine-tuning strategy that is both data-efficient and parameter-efficient. Anole demonstrates high-quality, coherent multimodal generation capabilities. We have open-sourced our model, training framework, and instruction tuning data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
