arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.15276cs.IRcs.AIcs.CV
收藏

CFIR:面向大型语料库的快速高效长文本到图像检索

CFIR: Fast and Effective Long-Text To Image Retrieval for Large Corpora

University of Glasgow(格拉斯哥大学)

Zijun Long, Xuri Ge, Richard Mccreadie, Joemon Jose

AI总结:

针对大规模长文本到图像检索中 MLLM 成本高、嵌入表达受限的问题,提出两阶段 CFIR 框架与 Decoupling-BEiT-3 编码器,显著提升召回率并大幅降低训练和检索耗时。

摘要:

文本到图像检索旨在根据文本查询查找相关图像,在数字图书馆、电子商务和多媒体数据库等多种用例中具有重要意义。尽管多模态大语言模型(MLLM)展现出最先进的性能,但由于计算成本及其生成的单射嵌入,它们在处理大规模、多样化且模糊的真实检索需求时存在局限。本文提出一种两阶段的由粗到细索引共享检索(Coarse-to-Fine Index-shared Retrieval,CFIR)框架,专为快速、高效的大规模长文本到图像检索而设计。第一阶段为基于实体的排序(Entity-based Ranking,ER),采用多查询到多目标范式以适应长文本查询的模糊性,从而为下一阶段进行候选过滤。第二阶段为基于摘要的重排序(Summary-based Re-ranking,SR),利用摘要化查询细化这些排序。我们还提出专门的 Decoupling-BEiT-3 编码器,针对处理模糊用户需求和两个阶段进行优化,并通过基于向量的相似度推理提升计算效率。在 AToMiC 数据集上的评估表明,CFIR 在 Recall@1000 上最高超过现有 MLLM 11.06%,同时将训练时间和检索时间分别减少 68.75% 和 99.79%。我们将在 https://github.com/longkukuhi/CFIR 发布代码,以促进未来研究。

英文摘要:

Text-to-image retrieval aims to find the relevant images based on a text query, which is important in various use-cases, such as digital libraries, e-commerce, and multimedia databases. Although Multimodal Large Language Models (MLLMs) demonstrate state-of-the-art performance, they exhibit limitations in handling large-scale, diverse, and ambiguous real-world needs of retrieval, due to the computation cost and the injective embeddings they produce. This paper presents a two-stage Coarse-to-Fine Index-shared Retrieval (CFIR) framework, designed for fast and effective large-scale long-text to image retrieval. The first stage, Entity-based Ranking (ER), adapts to long-text query ambiguity by employing a multiple-queries-to-multiple-targets paradigm, facilitating candidate filtering for the next stage. The second stage, Summary-based Re-ranking (SR), refines these rankings using summarized queries. We also propose a specialized Decoupling-BEiT-3 encoder, optimized for handling ambiguous user needs and both stages, which also enhances computational efficiency through vector-based similarity inference. Evaluation on the AToMiC dataset reveals that CFIR surpasses existing MLLMs by up to 11.06% in Recall@1000, while reducing training and retrieval times by 68.75% and 99.79%, respectively. We will release our code to facilitate future research at https://github.com/longkukuhi/CFIR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑