指示性图像检索：将黑盒学习变为灰色
Indicative Image Retrieval: Turning Blackbox Learning into Grey
摘要：
深度学习在引入后不久就成为图像检索的游戏规则改变者。该算法将特征提取（通过表示学习）作为图像检索的核心，将相关性/匹配性评价退化为简单的相似性度量。在许多应用中，我们需要指示匹配的证据，而不仅仅是排名列表（例如，医学图像中目标蛋白质/细胞/损伤的位置）。这就像匹配的单词需要在搜索引擎中突出显示一样。然而，如果没有明确的相关性/匹配建模，这是不容易实现的。深度表示学习模型由于其黑盒性质而不可行。在本文中，我们重新审视了相关性/匹配建模在深度学习时代的重要性，并采用了指示性检索设置。研究表明，跳过表示学习直接对匹配证据进行建模是可能的。通过消除对预训练模型的依赖性，它避免了许多相关问题（例如，分类和检索之间的域间隙、卷积引起的细节扩散等）。更重要的是，该研究表明，匹配可以显式建模和回溯后生成匹配的证据指示。它可以提高深层推理的可解释性。我们的方法在Oxford-5 k和Paris-6 k上都获得了文献中的最佳性能，并在Oxford-5 k上创下了97.77%的新纪录（Paris-6 k上为97.81%），而没有提取任何深度特征。
英文摘要：
Deep learning became the game changer for image retrieval soon after it was introduced. It promotes the feature extraction (by representation learning) as the core of image retrieval, with the relevance/matching evaluation being degenerated into simple similarity metrics. In many applications, we need the matching evidence to be indicated rather than just have the ranked list (e.g., the locations of the target proteins/cells/lesions in medical images). It is like the matched words need to be highlighted in search engines. However, this is not easy to implement without explicit relevance/matching modeling. The deep representation learning models are not feasible because of their blackbox nature. In this paper, we revisit the importance of relevance/matching modeling in deep learning era with an indicative retrieval setting. The study shows that it is possible to skip the representation learning and model the matching evidence directly. By removing the dependency on the pre-trained models, it has avoided a lot of related issues (e.g., the domain gap between classification and retrieval, the detail-diffusion caused by convolution, and so on). More importantly, the study demonstrates that the matching can be explicitly modeled and backtracked later for generating the matching evidence indications. It can improve the explainability of deep inference. Our method obtains a best performance in literature on both Oxford-5k and Paris-6k, and sets a new record of 97.77% on Oxford-5k (97.81% on Paris-6k) without extracting any deep features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
