GroundVLP：利用视觉-语言预训练和开放词汇目标检测实现零样本视觉定位
GroundVLP: Harnessing Zero-shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection
AI总结：
GroundVLP提出一种零样本视觉定位方法，通过融合GradCAM热力图与开放词汇检测器目标提议，在RefCOCO等数据集上显著超越现有零样本方法，性能提升约28%。
摘要：
视觉定位是一项关键的视觉-语言任务，涉及根据查询表达式理解视觉上下文，要求模型捕捉对象之间的交互以及各种空间和属性信息。然而，由于视觉定位任务的标注过程耗时且费力，其标注数据有限，导致训练出的模型难以将其能力泛化到更广泛的领域。为应对这一挑战，我们提出了GroundVLP，一种简单而有效的零样本方法，它从现有模型中提取视觉定位能力，这些模型基于图像-文本对和纯目标检测数据训练，与视觉定位标注数据相比，这些数据更容易获取且覆盖领域更广。GroundVLP提出了一种融合机制，结合了GradCAM的热力图和开放词汇检测器的目标提议。我们证明，所提方法在RefCOCO/+/g数据集上显著优于其他零样本方法，在RefCOCO和RefCOCO+的测试集上，相较于先前的零样本最先进方法，性能提升约28%。此外，在Flickr30k实体数据集上，GroundVLP的表现与一些非基于VLP的监督模型相当，甚至更好。我们的代码可在https://github.com/om-ai-lab/GroundVLP获取。
英文摘要：
Visual grounding, a crucial vision-language task involving the understanding of the visual context based on the query expression, necessitates the model to capture the interactions between objects, as well as various spatial and attribute information. However, the annotation data of visual grounding task is limited due to its time-consuming and labor-intensive annotation process, resulting in the trained models being constrained from generalizing its capability to a broader domain. To address this challenge, we propose GroundVLP, a simple yet effective zero-shot method that harnesses visual grounding ability from the existing models trained from image-text pairs and pure object detection data, both of which are more conveniently obtainable and offer a broader domain compared to visual grounding annotation data. GroundVLP proposes a fusion mechanism that combines the heatmap from GradCAM and the object proposals of open-vocabulary detectors. We demonstrate that the proposed method significantly outperforms other zero-shot methods on RefCOCO/+/g datasets, surpassing prior zero-shot state-of-the-art by approximately 28\% on the test split of RefCOCO and RefCOCO+. Furthermore, GroundVLP performs comparably to or even better than some non-VLP-based supervised models on the Flickr30k entities dataset. Our code is available at https://github.com/om-ai-lab/GroundVLP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
