arXiv 2602.016392026-03-31cs.CV

ReCALL: 为基于MLLM的组合图像检索 recalibrate 能力退化

ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval

Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Southeast University（东南大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；National University of Singapore（新加坡国立大学）；Wuhan AI Research（武汉人工智能研究院）；Guangdong Provincial Key Laboratory of Intellectual Property and Big Data, Guangdong Polytechnic Normal University（广东技术师范大学广东省知识产权大数据重点实验室）

Tianyu Yang, Chenwei He, Xiangzhao Hao, Tianyue Wang, Jiarui Guo, Haiyun Guo, Leigang Qu, Jinqiao Wang, Tat-Seng Chua

备注：: Accepted to CVPR 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ReCALL通过诊断生成器盲点、生成修正指令和三元组、并持续训练来缓解基于生成式MLLM的检索能力退化问题，实验证明其在CIRR和FashionIQ上达到SOTA性能。

摘要：

组合图像检索（CIR）旨在通过参考图像和修改文本的混合查询检索目标图像。早期双塔视觉-语言模型（VLMs）在处理此任务所需的跨模态组合推理方面存在困难。尽管将生成式多模态大语言模型（MLLMs）适应于检索是一个有前途的方向，但我们发现这种策略忽略了根本问题：将生成式MLLM压缩成单嵌入判别检索器会引发范式冲突，导致能力退化——检索适应后原生细粒度推理能力的下降。为了解决这一挑战，我们提出了ReCALL，一个模型无关的框架，遵循诊断-生成-细化流程：首先通过自引导信息性实例挖掘诊断检索器的认知盲点。接着通过提示基础MLLM生成修正指令和三元组，并通过VQA一致性过滤进行质量控制。最后通过持续训练这些三元组，使用分组对比方案优化检索器，从而内化细粒度视觉-语义差异，并将检索器的判别嵌入空间与MLLM内在的组合推理重新对齐。在CIRR和FashionIQ上的大量实验表明，ReCALL一致地 recalibrates 退化的能力，并实现了SOTA性能。代码可在https://github.com/RemRico/Recall中获得。

英文摘要：

Composed Image Retrieval (CIR) aims to retrieve target images based on a hybrid query comprising a reference image and a modification text. Early dual-tower Vision-Language Models (VLMs) struggle with cross-modality compositional reasoning required for this task. While adapting generative Multimodal Large Language Models (MLLMs) for retrieval offers a promising direction, we identify that this strategy overlooks a fundamental issue: compressing a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation - the deterioration of native fine-grained reasoning after retrieval adaptation. To address this challenge, we propose ReCALL, a model-agnostic framework that follows a diagnose-generate-refine pipeline: First, we diagnose cognitive blind spots of the retriever via self-guided informative instance mining. Next, we generate corrective instructions and triplets by prompting the foundation MLLM and conduct quality control with VQA-based consistency filtering. Finally, we refine the retriever through continual training on these triplets with a grouped contrastive scheme, thereby internalizing fine-grained visual-semantic distinctions and realigning the discriminative embedding space of retriever with intrinsic compositional reasoning within the MLLM. Extensive experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance. Code is available at https://github.com/RemRico/Recall.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。