LanteRn：潜在视觉结构推理
LanteRn: Latent Visual Structured Reasoning
AI总结：
本文提出LanteRn框架，通过将语言与紧凑的潜在视觉表示结合，使大多模态模型能在潜在空间中进行视觉推理，提升视觉理解和细粒度推理能力。
摘要：
尽管语言推理模型在许多任务中表现优异，但当前大型多模态模型（LMMs）在视觉推理方面仍面临挑战。因此，大多数LMMs倾向于将感知内容转化为文本，这在需要精细空间和视觉理解的任务中存在明显局限。尽管近期方法通过调用工具或生成中间图像来尝试图像思考，但它们要么依赖外部模块，要么通过在像素空间中直接推理导致计算开销。本文介绍LanteRn，一个使LMMs能够将语言与紧凑的潜在视觉表示交织的框架，使视觉推理可以直接在潜在空间中进行。LanteRn增强了视觉-语言转换器，在推理过程中生成并关注连续的视觉思维嵌入。我们通过两个阶段训练模型：监督微调以在潜在状态中定位视觉特征，随后通过强化学习将潜在推理与任务层面的效用对齐。我们在三个以感知为中心的基准（VisCoT、V*和Blink）上评估LanteRn，观察到在视觉定位和细粒度推理方面的一致改进。这些结果表明，内部潜在表示为更高效的多模态推理提供了一个有前景的方向。
英文摘要：
While language reasoning models excel in many tasks, visual reasoning remains challenging for current large multimodal models (LMMs). As a result, most LMMs default to verbalizing perceptual content into text, a strong limitation for tasks requiring fine-grained spatial and visual understanding. While recent approaches take steps toward thinking with images by invoking tools or generating intermediate images, they either rely on external modules, or incur unnecessary computation by reasoning directly in pixel space. In this paper, we introduce LanteRn, a framework that enables LMMs to interleave language with compact latent visual representations, allowing visual reasoning to occur directly in latent space. LanteRn augments a vision-language transformer with the ability to generate and attend to continuous visual thought embeddings during inference. We train the model in two stages: supervised fine-tuning to ground visual features in latent states, followed by reinforcement learning to align latent reasoning with task-level utility. We evaluate LanteRn on three perception-centric benchmarks (VisCoT, V*, and Blink), observing consistent improvements in visual grounding and fine-grained reasoning. These results suggest that internal latent representations provide a promising direction for more efficient multimodal reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
