EVEv2：无编码器视觉-语言模型的改进基线
EVEv2: Improved Baselines for Encoder-Free Vision-Language Models
AI总结：
本文针对无编码器视觉-语言模型，分析不同视觉建模路径的性能差距，提出模态解耦、层级关联与训练优化策略，并发布改进基线 EVEv2.0。
摘要：
现有的无编码器视觉-语言模型（VLM）正在迅速缩小其与基于编码器的同类模型之间的性能差距，凸显出构建兼具结构简洁性与高效部署能力的统一多模态系统的良好潜力。我们系统地阐明了使用预训练视觉编码器、离散 tokenizer 以及从零开始的极简视觉层的 VLM 之间的性能差距，深入挖掘了无编码器 VLM 中尚未得到充分研究的特性。我们开发了可与主流基于编码器的 VLM 相媲美的无编码器 VLM 高效策略。经过深入研究，我们推出了 EVEv2.0，这是一个全新且经过改进的无编码器 VLM 系列。我们表明：（i）在统一模型内适当地分解视觉与语言并建立层级关联，可减少模态之间的干扰。（ii）设计良好的训练策略能够实现对无编码器 VLM 的有效优化。通过广泛评估，我们的 EVEv2.0 代表了对开发跨模态 decoder-only 架构的全面研究，展现出卓越的数据效率和强大的视觉推理能力。代码公开发布于：https://github.com/baaivision/EVE。
英文摘要：
Existing encoder-free vision-language models (VLMs) are rapidly narrowing the performance gap with their encoder-based counterparts, highlighting the promising potential for unified multimodal systems with structural simplicity and efficient deployment. We systematically clarify the performance gap between VLMs using pre-trained vision encoders, discrete tokenizers, and minimalist visual layers from scratch, deeply excavating the under-examined characteristics of encoder-free VLMs. We develop efficient strategies for encoder-free VLMs that rival mainstream encoder-based ones. After an in-depth investigation, we launch EVEv2.0, a new and improved family of encoder-free VLMs. We show that: (i) Properly decomposing and hierarchically associating vision and language within a unified model reduces interference between modalities. (ii) A well-designed training strategy enables effective optimization for encoder-free VLMs. Through extensive evaluation, our EVEv2.0 represents a thorough study for developing a decoder-only architecture across modalities, demonstrating superior data efficiency and strong vision-reasoning capability. Code is publicly available at: https://github.com/baaivision/EVE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
