长尾视觉识别中的检索增强分类
Retrieval Augmented Classification for Long-Tail Visual Recognition
摘要：
我们介绍检索增强分类（RAC），一个通用的方法来增强标准的图像分类管道与一个明确的检索模块。RAC由一个标准的基本图像编码器与一个并行检索分支组成，该分支查询预编码图像和相关文本片段的非参数外部存储器。我们将RAC应用于长尾分类问题，并在Places 365-LT和iNaturalist-2018上展示了比以前最先进的方法（分别为14.5%和6.7%）的显着改进，尽管仅使用训练数据集本身作为外部信息源。我们证明，RAC的检索模块，没有提示，学习高水平的准确性尾类。这又使基本编码器能够专注于公共类，并提高其性能。RAC代表了一种无需微调即可利用大型预训练模型的替代方法，也是在常见计算机视觉架构中更有效地利用外部存储器的第一步。
英文摘要：
We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module. RAC consists of a standard base image encoder fused with a parallel retrieval branch that queries a non-parametric external memory of pre-encoded images and associated text snippets. We apply RAC to the problem of long-tail classification and demonstrate a significant improvement over previous state-of-the-art on Places365-LT and iNaturalist-2018 (14.5% and 6.7% respectively), despite using only the training datasets themselves as the external information source. We demonstrate that RAC's retrieval module, without prompting, learns a high level of accuracy on tail classes. This, in turn, frees the base encoder to focus on common classes, and improve its performance thereon. RAC represents an alternative approach to utilizing large, pretrained models without requiring fine-tuning, as well as a first step towards more effectively making use of external memory within common computer vision architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
