arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.07022cs.IR
收藏

利用分布约束实现可扩展且高效的图像检索

Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval

Mohammad Omama, Po-han Li, Sandeep P. Chinchali

AI总结:

针对图像检索的可扩展性与效率挑战,提出AE-SVC方法优化基础模型的嵌入分布,并引入(SS)2D实现自适应嵌入大小,在多个数据集上显著提升了检索性能与效率。

摘要:

图像检索在机器人和计算机视觉中至关重要,在机器人位置识别和基于视觉的产品推荐等下游应用中均有应用。现代检索系统面临两个关键挑战:可扩展性和效率。最先进的图像检索系统为每个数据集训练特定的神经网络,这种方法缺乏可扩展性。此外,由于检索速度与嵌入大小成正比,使用大型嵌入的现有系统缺乏效率。为了解决可扩展性问题,最近的研究提出使用现成的基础模型。然而,这些模型虽然适用于跨数据集,但在实现与数据集特定模型相当的性能方面仍有不足。我们的关键观察是,虽然基础模型捕捉了有效检索所需的细微特征,但其嵌入空间的底层分布可能会对余弦相似度搜索产生负面影响。我们引入了具有强方差约束的自编码器(AE-SVC),当用于投影时,它能显著提升基础模型的性能。我们对AE-SVC提供了深入的理论分析。针对效率问题,我们引入了单次相似度空间蒸馏((SS)$_2$D),这是一种学习自适应大小嵌入的新方法,在大小和性能之间提供了更好的权衡。我们在四个检索数据集上进行了广泛实验,包括Stanford Online Products (SoP)和Pittsburgh30k,使用了四个不同的现成基础模型,包括DinoV2和CLIP。AE-SVC在检索性能上实现了高达16%的提升,而(SS)$_2$D在更小的嵌入大小下进一步展现了10%的提升。

英文摘要:

Image retrieval is crucial in robotics and computer vision, with downstream applications in robot place recognition and vision-based product recommendations. Modern retrieval systems face two key challenges: scalability and efficiency. State-of-the-art image retrieval systems train specific neural networks for each dataset, an approach that lacks scalability. Furthermore, since retrieval speed is directly proportional to embedding size, existing systems that use large embeddings lack efficiency. To tackle scalability, recent works propose using off-the-shelf foundation models. However, these models, though applicable across datasets, fall short in achieving performance comparable to that of dataset-specific models. Our key observation is that, while foundation models capture necessary subtleties for effective retrieval, the underlying distribution of their embedding space can negatively impact cosine similarity searches. We introduce Autoencoders with Strong Variance Constraints (AE-SVC), which, when used for projection, significantly improves the performance of foundation models. We provide an in-depth theoretical analysis of AE-SVC. Addressing efficiency, we introduce Single-shot Similarity Space Distillation ((SS)$_2$D), a novel approach to learn embeddings with adaptive sizes that offers a better trade-off between size and performance. We conducted extensive experiments on four retrieval datasets, including Stanford Online Products (SoP) and Pittsburgh30k, using four different off-the-shelf foundation models, including DinoV2 and CLIP. AE-SVC demonstrates up to a $16\%$ improvement in retrieval performance, while (SS)$_2$D shows a further $10\%$ improvement for smaller embedding sizes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑