arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05359cs.CV

多模态潜在推理 via 分层视觉提示注入

Multimodal Latent Reasoning via Hierarchical Visual Cues Injection

Nanyang Technological University(南洋理工大学);Huawei Noah's Ark Lab(华为诺亚实验室)

Yiming Zhang, Qiangyu Yan, Borui Jiang, Kai Han

AI总结:

本文提出HIVE框架,通过分层视觉提示注入实现多模态潜在推理,提升模型在对齐潜在空间中的多步推理能力。

摘要:

多模态大语言模型(MLLMs)的进步使感知能力显著提升,但其推理过程仍依赖于端到端生成或显式语言中心的推理链(CoT),效率低且易产生幻觉。本文提出HIVE框架,通过分层视觉提示注入,使推理过程在潜在空间中稳健进行。方法递归扩展transformer块,创建内部循环以迭代优化推理。关键在于将分层视觉提示从全局场景上下文注入到模型的潜在表示中,使模型能在对齐的潜在空间中进行基于现实的多步推理。大量评估显示,结合视觉知识时测试时扩展有效,整合分层信息显著提升模型对复杂场景的理解。

英文摘要:

The advancement of multimodal large language models (MLLMs) has enabled impressive perception capabilities. However, their reasoning process often remains a "fast thinking" paradigm, reliant on end-to-end generation or explicit, language-centric chains of thought (CoT), which can be inefficient, verbose, and prone to hallucination. This work posits that robust reasoning should evolve within a latent space, integrating multimodal signals seamlessly. We propose multimodal latent reasoning via HIerarchical Visual cuEs injection (\emph{HIVE}), a novel framework that instills deliberate, "slow thinking" without depending on superficial textual rationales. Our method recursively extends transformer blocks, creating an internal loop for iterative reasoning refinement. Crucially, it injectively grounds this process with hierarchical visual cues from global scene context to fine-grained regional details directly into the model's latent representations. This enables the model to perform grounded, multi-step inference entirely in the aligned latent space. Extensive evaluations demonstrate that test-time scaling is effective when incorporating vision knowledge, and that integrating hierarchical information significantly enhances the model's understanding of complex scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑