DiG：通过差异 grounding 提升多模态大语言模型的细粒度感知
DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model
AI总结：
本文提出DiG框架，通过学习相似图像对的差异识别提升多模态大语言模型的细粒度感知能力，实验表明其在多个视觉感知基准上表现优异。
摘要：
多模态大语言模型在多种视觉-语言任务中表现出色，但其细粒度视觉感知和精确空间推理仍有限。本文引入DiG（Differential Grounding），一种新的代理任务框架，使MLLMs通过识别和定位相似图像对之间的所有差异来学习细粒度感知，无需先验知识。为支持可扩展训练，我们开发了基于自动3D渲染的数据生成流水线，生成高质量配对图像并完全可控差异。为解决差异信号稀疏性问题，我们进一步采用课程学习，逐步增加从单个到多个差异的复杂度，实现稳定优化。大量实验表明，DiG在多种视觉感知基准上显著提高了模型性能，且所学的细粒度感知能力有效转移到标准下游任务，包括RefCOCO、RefCOCO+、RefCOCOg和通用多模态感知基准。结果表明，差异 grounding 是一种可扩展且稳健的方法，用于推进MLLMs的细粒度视觉推理。
英文摘要：
Multimodal Large Language Models have achieved impressive performance on a variety of vision-language tasks, yet their fine-grained visual perception and precise spatial reasoning remain limited. In this work, we introduce DiG (Differential Grounding), a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number. To support scalable training, we develop an automated 3D rendering-based data generation pipeline that produces high-quality paired images with fully controllable discrepancies. To address the sparsity of difference signals, we further employ curriculum learning that progressively increases complexity from single to multiple differences, enabling stable optimization. Extensive experiments demonstrate that DiG significantly improves model performance across a variety of visual perception benchmarks and that the learned fine-grained perception skills transfer effectively to standard downstream tasks, including RefCOCO, RefCOCO+, RefCOCOg, and general multimodal perception benchmarks. Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
