机器心理意象：利用潜在视觉标记增强多模态推理
Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens
AI总结：
提出Mirage框架，用潜在视觉标记增强VLM解码，通过蒸馏与强化学习实现无需显式图像的多模态推理，提升视觉想象任务性能。
摘要：
视觉语言模型（VLMs）在多模态理解方面表现出色，然而其纯文本解码方式迫使其将视觉推理过程言语化，这限制了它们在需要视觉想象的任务上的表现。近期尝试训练VLMs生成显式图像的方法，但繁重的图像生成预训练往往阻碍了推理能力。受人类通过心理意象（即视觉线索的内部构建与操作）进行推理的方式启发，我们探究VLMs是否能够通过交错的多模态轨迹进行推理，而无需生成显式图像。为此，我们提出了一种名为Mirage的机器心理意象框架，该框架在普通文本之外，用潜在视觉标记增强VLM的解码过程。具体而言，每当模型选择“视觉思考”时，它会将其隐藏状态重新转化为下一个标记，从而在不生成像素级图像的情况下继续多模态轨迹。我们首先通过从真实图像嵌入中蒸馏来监督潜在标记，随后切换到纯文本监督，使潜在轨迹与任务目标紧密对齐。后续的强化学习阶段进一步增强了多模态推理能力。在多样化基准上的实验表明，Mirage无需显式图像生成即可解锁更强的多模态推理能力。
英文摘要：
Vision-language models (VLMs) excel at multimodal understanding, yet their text-only decoding forces them to verbalize visual reasoning, limiting performance on tasks that demand visual imagination. Recent attempts train VLMs to render explicit images, but the heavy image-generation pre-training often hinders the reasoning ability. Inspired by the way humans reason with mental imagery-the internal construction and manipulation of visual cues-we investigate whether VLMs can reason through interleaved multimodal trajectories without producing explicit images. To this end, we present a Machine Mental Imagery framework, dubbed as Mirage, which augments VLM decoding with latent visual tokens alongside ordinary text. Concretely, whenever the model chooses to ``think visually'', it recasts its hidden states as next tokens, thereby continuing a multimodal trajectory without generating pixel-level images. Begin by supervising the latent tokens through distillation from ground-truth image embeddings, we then switch to text-only supervision to make the latent trajectory align tightly with the task objective. A subsequent reinforcement learning stage further enhances the multimodal reasoning capability. Experiments on diverse benchmarks demonstrate that Mirage unlocks stronger multimodal reasoning without explicit image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
