arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.03307cs.CVcs.AI
收藏

V-Reflection:将多模态大语言模型从被动观察者转变为主动提问者

V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators

AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州)人工智能方向);International Digital Economy Academy(国际数字经济学院);MedVisAI Lab, Lee Kong Chian School of Medicine, Nanyang Technological University, and Centre of AI in Medicine(医学视觉人工智能实验室,南洋理工大学Lee Kong Chian医学院,以及人工智能在医学中的中心);South China University of Technology(华南理工大学);Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(香港理工大学电子与电气工程系)

Jiazhou Zhou, Yucheng Chen, Hongyang Li, Qing Jiang, Hu Zhou, Ying-Cong Chen, Lei Zhang

AI总结:

本文提出V-Reflection框架,通过'思考后再观察'的视觉反思机制,使多模态大语言模型能主动提问视觉特征空间,提升细粒度任务的感知能力。

摘要:

多模态大语言模型(MLLMs)在细粒度任务中易产生感知幻觉,因其推理受限于语言领域,将视觉输入视为静态前奏而非动态参与者。为此,本文提出V-Reflection框架,通过'思考后再观察'的视觉反思机制,将MLLM转变为主动提问者。该框架采用两阶段蒸馏策略:首先,Box-Guided Compression Module(BCM)通过显式空间定位建立稳定的像素到潜在状态目标;其次,Dynamic Autoregressive Compression(DAC)模块将模型的隐藏状态映射为动态探针,用于查询全局视觉特征图。通过将BCM教师的视觉空间专业知识蒸馏到DAC学生中,V-Reflection内部化了定位任务关键证据的能力。在推理过程中,两个模块保持完全不活动,维持潜在空间中的纯端到端自回归解码,以最优效率运行。大量实验表明,V-Reflection在六个感知密集基准上均表现出色,显著缩小了细粒度感知差距。可视化结果确认,潜在推理能自主定位任务关键的视觉证据。

英文摘要:

Multimodal Large Language Models (MLLMs) have achieved remarkable success, yet they remain prone to perception-related hallucinations in fine-grained tasks. This vulnerability arises from a fundamental limitation: their reasoning is largely restricted to the language domain, treating visual input as a static, reasoning-agnostic preamble rather than a dynamic participant. Consequently, current models act as passive observers, unable to re-examine visual details to ground their evolving reasoning states. To overcome this, we propose V-Reflection, a framework that transforms the MLLM into an active interrogator through a "think-then-look" visual reflection mechanism. During reasoning, latent states function as dynamic probes that actively interrogate the visual feature space, grounding each reasoning step for task-critical evidence. Our approach employs a two-stage distillation strategy. First, the Box-Guided Compression Module (BCM) establishes stable pixel-to-latent targets through explicit spatial grounding. Next, a Dynamic Autoregressive Compression (DAC) module maps the model's hidden states into dynamic probes that interrogate the global visual feature map. By distilling the spatial expertise of the BCM teacher into the DAC student, V-Reflection internalizes the ability to localize task-critical evidence. During inference, both modules remain entirely inactive, maintaining a purely end-to-end autoregressive decoding in the latent space with optimal efficiency. Extensive experiments demonstrate the effectiveness of our V-Reflection across six perception-intensive benchmarks, significantly narrowing the fine-grained perception gap. Visualizations confirm that latent reasoning autonomously localizes task-critical visual evidence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑