GeoSense: 内化几何必要性感知以实现多模态推理
GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning
AI总结：
GeoSense通过内化几何必要性感知，提升多模态推理的鲁棒性和效率。
摘要：
迈向人工超级智能需要丰富的智能感知能力。在这个追求过程中，一个关键前沿是克服多模态大语言模型（MLLMs）的有限空间理解能力，其中几何信息是至关重要的。现有方法通常通过将几何信号严格注入到每一个输入中来解决这个问题，而忽略了其必要性并增加了计算开销。与这一范式相反，我们的框架赋予模型感知不足的意识，使其能够在2D线索被认为不足时自主地参与几何特征的推理。为此，我们首先将独立的几何输入通道引入模型架构，并进行对齐训练，以有效地利用几何特征。随后，为了赋予模型感知意识，我们精心挑选了一个专门的空间感知监督微调数据集。这有助于激活模型的潜在内部线索，使其能够自主决定几何信息的必要性。在多个空间推理基准测试中进行的实验验证了这一方法，证明了在不损害2D视觉推理能力的情况下，取得了显著的空间增益，为更稳健、高效和自知的多模态智能提供了一条道路。
英文摘要：
Advancing towards artificial superintelligence requires rich and intelligent perceptual capabilities. A critical frontier in this pursuit is overcoming the limited spatial understanding of Multimodal Large Language Models (MLLMs), where geometry information is essential. Existing methods often address this by rigidly injecting geometric signals into every input, while ignoring their necessity and adding computation overhead. Contrary to this paradigm, our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient. To achieve this, we first introduce an independent geometry input channel to the model architecture and conduct alignment training, enabling the effective utilization of geometric features. Subsequently, to endow the model with perceptual awareness, we curate a dedicated spatial-aware supervised fine-tuning dataset. This serves to activate the model's latent internal cues, empowering it to autonomously determine the necessity of geometric information. Experiments across multiple spatial reasoning benchmarks validate this approach, demonstrating significant spatial gains without compromising 2D visual reasoning capabilities, offering a path toward more robust, efficient and self-aware multi-modal intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
