像阅读文本一样理解图像：视觉-语言模型中的序列图像理解
Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
AI总结：
本文提出通过序列化方法理解图像内容，揭示视觉-语言模型中对象识别与空间感知的机制，改进解码效率并增强空间推理能力。
摘要：
视觉-语言模型（VLMs）在各种现实任务中展现了显著的性能。然而，现有的VLMs通常通过串行化图像来处理视觉信息，这种方法与人类视觉的并行性质有显著差异。此外，其不透明的内部机制阻碍了更深入的理解和架构创新。受人类视觉双流假说的启发，该假说区分了“是什么”和“在哪里”的路径，我们将VLMs中的视觉处理分解为对象识别和空间感知进行单独研究。对于对象识别，我们将图像转换为文本标记地图，并发现模型对图像内容的感知展开为从浅层到深层的两阶段过程，从属性识别开始，最终达到语义消歧。对于空间感知，我们理论推导并经验验证了VLMs中位置表示下的几何结构。基于这些发现，我们引入了一种基于插件式视觉解码器的指令无关标记压缩算法，以提高解码效率，并引入RoPE缩放技术以增强空间推理。通过严谨的实验，我们的工作验证了这些分析，提供了对VLM内部更深入的理解，并为设计更具能力的未来架构提供了清晰的原则。
英文摘要：
Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks. However, existing VLMs typically process visual information by serializing images, a method that diverges significantly from the parallel nature of human vision. Moreover, their opaque internal mechanisms hinder both deeper understanding and architectural innovation. Inspired by the dual-stream hypothesis of human vision, which distinguishes the "what" and "where" pathways, we deconstruct the visual processing in VLMs into object recognition and spatial perception for separate study. For object recognition, we convert images into text token maps and find that the model's perception of image content unfolds as a two-stage process from shallow to deep layers, beginning with attribute recognition and culminating in semantic disambiguation. For spatial perception, we theoretically derive and empirically verify the geometric structure underlying the positional representation in VLMs. Based on these findings, we introduce an instruction-agnostic token compression algorithm based on a plug-and-play visual decoder to improve decoding efficiency, and a RoPE scaling technique to enhance spatial reasoning. Through rigorous experiments, our work validates these analyses, offering a deeper understanding of VLM internals and providing clear principles for designing more capable future architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
