语言引导的扩散模型用于视觉定位
Language-Guided Diffusion Model for Visual Grounding
AI总结：
提出语言引导扩散框架LG-DVG，通过迭代去噪噪声框实现视觉定位，在五个数据集上验证了生成式方法的优越性能。
摘要：
视觉定位（Visual Grounding, VG）任务涉及显式的跨模态对齐，即需要为给定的语言短语定位语义上对应的图像区域。现有方法以单步方式完成这种视觉-文本推理。它们的性能对大规模锚框和基于人类先验的过度设计的多模态融合模块有很高要求，导致框架复杂，可能难以训练且容易过拟合特定场景。更糟糕的是，这种一次性推理机制无法持续细化边界框以增强查询-区域匹配。相比之下，在本文中，我们通过去噪扩散建模构建了一个迭代推理过程。具体来说，我们提出了一个用于视觉定位的语言引导扩散框架LG-DVG，该框架训练模型通过语言引导对一组噪声边界框进行去噪，逐步推理查询的目标边界框。为实现这一点，LG-DVG逐步将查询对齐的真实边界框扰动为噪声框，并在查询语义的条件下逐步逆转这一过程。我们在五个广泛使用的数据集上对我们提出的框架进行了大量实验，验证了以生成方式解决视觉定位这一跨模态对齐任务的优越性能。源代码可在https://github.com/iQua/vgbase/tree/main/examples/DiffusionVG获取。
英文摘要：
Visual grounding (VG) tasks involve explicit cross-modal alignment, as semantically corresponding image regions are to be located for the language phrases provided. Existing approaches complete such visual-text reasoning in a single-step manner. Their performance causes high demands on large-scale anchors and over-designed multi-modal fusion modules based on human priors, leading to complicated frameworks that may be difficult to train and overfit to specific scenarios. Even worse, such once-for-all reasoning mechanisms are incapable of refining boxes continuously to enhance query-region matching. In contrast, in this paper, we formulate an iterative reasoning process by denoising diffusion modeling. Specifically, we propose a language-guided diffusion framework for visual grounding, LG-DVG, which trains the model to progressively reason queried object boxes by denoising a set of noisy boxes with the language guide. To achieve this, LG-DVG gradually perturbs query-aligned ground truth boxes to noisy ones and reverses this process step by step, conditional on query semantics. Extensive experiments for our proposed framework on five widely used datasets validate the superior performance of solving visual grounding, a cross-modal alignment task, in a generative way. The source codes are available at https://github.com/iQua/vgbase/tree/main/examples/DiffusionVG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
