ScanFormer：通过迭代扫描进行指代表达理解
ScanFormer: Referring Expression Comprehension by Iteratively Scanning
AI总结：
ScanFormer提出从粗到细的迭代扫描框架，通过图像尺度金字塔和补丁丢弃策略消除与语言无关的冗余视觉区域，在四个数据集上实现准确性与效率的平衡。
摘要：
指代表达理解（REC）旨在定位图像中由自由形式自然语言描述指定的目标对象。尽管最先进的方法取得了令人印象深刻的性能，但它们对图像进行密集感知，其中包含与语言查询无关的冗余视觉区域，导致额外的计算开销。这启发我们探索一个问题：我们能否消除与语言无关的冗余视觉区域以提高模型的效率？现有的相关方法主要关注基础视觉任务，在视觉-语言领域的探索有限。为了解决这个问题，我们提出了一种从粗到细的迭代感知框架，称为ScanFormer。它可以迭代地利用图像尺度金字塔，从上到下提取与语言相关的视觉补丁。在每次迭代中，通过我们设计的信息性预测丢弃不相关的补丁。此外，我们提出了一种针对被丢弃补丁的补丁选择策略以加速推理。在广泛使用的数据集，即RefCOCO、RefCOCO+、RefCOCOg和ReferItGame上的实验验证了我们方法的有效性，它可以在准确性和效率之间取得平衡。
英文摘要：
Referring Expression Comprehension (REC) aims to localize the target objects specified by free-form natural language descriptions in images. While state-of-the-art methods achieve impressive performance, they perform a dense perception of images, which incorporates redundant visual regions unrelated to linguistic queries, leading to additional computational overhead. This inspires us to explore a question: can we eliminate linguistic-irrelevant redundant visual regions to improve the efficiency of the model? Existing relevant methods primarily focus on fundamental visual tasks, with limited exploration in vision-language fields. To address this, we propose a coarse-to-fine iterative perception framework, called ScanFormer. It can iteratively exploit the image scale pyramid to extract linguistic-relevant visual patches from top to bottom. In each iteration, irrelevant patches are discarded by our designed informativeness prediction. Furthermore, we propose a patch selection strategy for discarded patches to accelerate inference. Experiments on widely used datasets, namely RefCOCO, RefCOCO+, RefCOCOg, and ReferItGame, verify the effectiveness of our method, which can strike a balance between accuracy and efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
