arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.12236cs.CV
收藏

基于重排Transformers的实例级图像检索

Instance-level Image Retrieval using Reranking Transformers

University of Virginia(弗吉尼亚大学);eBay Computer Vision(eBay计算机视觉部门);Rice University(莱斯大学)

Fuwen Tan, Jiangbo Yuan, Vicente Ordonez

摘要:

实例级图像检索是在大型数据库中搜索与查询图像中的对象相匹配的图像的任务。为了解决这一任务,系统通常依赖于使用全局图像描述符的检索步骤,以及通过利用诸如基于局部特征的几何验证之类的操作来执行特定于域的细化或重新排序的后续步骤。在这项工作中,我们提出了重新排序Transformers(RRTs)作为一个通用的模型,将本地和全球的功能重新排序的匹配图像在监督的方式,从而取代相对昂贵的过程中的几何验证。RRT是轻量级的,并且可以容易地并行化,使得可以在单个向前传递中执行对一组顶级匹配结果的重新排序。我们在Revisited Oxford和Paris数据集以及Google Landmarks v2数据集上进行了广泛的实验,结果表明RRT在使用更少的局部描述符的同时优于以前的重新排名方法。此外,我们证明,与现有的方法不同,RRT可以与特征提取器联合优化,这可以导致针对下游任务的特征表示和进一步的准确性提高。代码和训练模型可在https://github.com/uvavision/RerankingTransformer上公开获得。

英文摘要:

Instance-level image retrieval is the task of searching in a large database for images that match an object in a query image. To address this task, systems usually rely on a retrieval step that uses global image descriptors, and a subsequent step that performs domain-specific refinements or reranking by leveraging operations such as geometric verification based on local features. In this work, we propose Reranking Transformers (RRTs) as a general model to incorporate both local and global features to rerank the matching images in a supervised fashion and thus replace the relatively expensive process of geometric verification. RRTs are lightweight and can be easily parallelized so that reranking a set of top matching results can be performed in a single forward-pass. We perform extensive experiments on the Revisited Oxford and Paris datasets, and the Google Landmarks v2 dataset, showing that RRTs outperform previous reranking approaches while using much fewer local descriptors. Moreover, we demonstrate that, unlike existing approaches, RRTs can be optimized jointly with the feature extractor, which can lead to feature representations tailored to downstream tasks and further accuracy improvements. The code and trained models are publicly available at https://github.com/uvavision/RerankingTransformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑