基于自然语言的图像差异定位
Image Difference Grounding with Natural Language
AI总结：
提出图像差异定位（IDG）任务及大规模数据集 DiffGround，并开发基线模型 DiffTracker，通过特征差异增强与共同抑制精确定位多图像间的细粒度视觉差异。
摘要：
视觉定位（VG）通常侧重于使用自然语言定位单个图像内的感兴趣区域，且大多数现有 VG 方法仅限于单图像解释。这限制了它们在自动监控等现实场景中的适用性，在这些场景中，检测多张图像间细微但有意义的视觉差异至关重要。此外，以往关于图像差异理解（IDU）的工作要么集中于在没有跨模态文本引导的情况下检测所有变化区域，要么提供粗粒度的差异描述。因此，为了推动更细粒度的视觉-语言感知，我们提出了图像差异定位（IDG），这是一项旨在根据用户指令精确定位视觉差异的任务。我们引入了 DiffGround，一个用于 IDG 的大规模高质量数据集，包含具有多样视觉变化的图像对以及查询细粒度差异的指令。此外，我们提出了 IDG 的基线模型 DiffTracker，它有效集成了特征差异增强和共同抑制来精确定位差异。在 DiffGround 数据集上的实验突出了我们的 IDG 数据集在实现更细粒度 IDU 方面的重要性。为促进未来研究，DiffGround 数据和 DiffTracker 模型将公开发布。
英文摘要：
Visual grounding (VG) typically focuses on locating regions of interest within an image using natural language, and most existing VG methods are limited to single-image interpretations. This limits their applicability in real-world scenarios like automatic surveillance, where detecting subtle but meaningful visual differences across multiple images is crucial. Besides, previous work on image difference understanding (IDU) has either focused on detecting all change regions without cross-modal text guidance, or on providing coarse-grained descriptions of differences. Therefore, to push towards finer-grained vision-language perception, we propose Image Difference Grounding (IDG), a task designed to precisely localize visual differences based on user instructions. We introduce DiffGround, a large-scale and high-quality dataset for IDG, containing image pairs with diverse visual variations along with instructions querying fine-grained differences. Besides, we present a baseline model for IDG, DiffTracker, which effectively integrates feature differential enhancement and common suppression to precisely locate differences. Experiments on the DiffGround dataset highlight the importance of our IDG dataset in enabling finer-grained IDU. To foster future research, both DiffGround data and DiffTracker model will be publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
