arXiv 2604.072012026-04-08cs.IRcs.CV

BRIDGE：通过强化学习查询对齐实现多模态到文本检索

BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment

High institute for computer & information systems（高等计算机与信息系统学院）；Chungbuk National University（忠北大学）；Assiut University（艾斯尤特大学）；University of Innsbruck（因斯布鲁克大学）

Mohamed Darwish Mounis, Mohamed Mahmoud, Shaimaa Sedek, Mahmoud Abdalla, Mahmoud SalahEldin Kasem, Abdelrahman Abdallah, Hyun-Soo Kang

备注：: Accepted at CVPR 2026 Workshop GRAIL-V

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

BRIDGE通过强化学习查询对齐模型和增强神经搜索检索器，解决多模态查询在文本库中的检索问题，实现优于多模态编码器的nDCG@10性能。

摘要：

多模态检索系统在处理图像-文本查询时难以有效检索文本库：最佳视觉语言编码器在MM-BRIGHT数据集上仅能达到27.6 nDCG@10，低于强文本检索器。我们提出BRIDGE系统，该系统不依赖多模态编码器即可解决这一问题。FORGE是通过强化学习训练的查询对齐模型，将嘈杂的多模态查询压缩为紧凑的检索优化搜索字符串。LENS是经过推理密集检索数据微调的密集检索器，以处理FORGE生成的意图丰富的查询。在MM-BRIGHT数据集（2,803个查询，29个领域）上评估，BRIDGE达到29.7 nDCG@10，超过所有多模态编码器基线，包括Nomic-Vision（27.6）。当FORGE作为插件在Nomic-Vision上应用时，联合系统达到33.3 nDCG@10，超过最佳文本检索器（32.2），证明查询对齐是多模态到文本检索的关键瓶颈。

英文摘要：

Multimodal retrieval systems struggle to resolve image-text queries against text-only corpora: the best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers. We argue the bottleneck is not the retriever but the query -- raw multimodal queries entangle visual descriptions, conversational noise, and retrieval intent in ways that systematically degrade embedding similarity. We present \textbf{BRIDGE}, a two-component system that resolves this mismatch without multimodal encoders. \textbf{FORGE} (\textbf{F}ocused Retrieval Query Generato\textbf{r}) is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings. \textbf{LENS} (\textbf{L}anguage-\textbf{E}nhanced \textbf{N}eural \textbf{S}earch) is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces. Evaluated on MM-BRIGHT (2,803 queries, 29 domains), BRIDGE achieves \textbf{29.7} nDCG@10, surpassing all multimodal encoder baselines including Nomic-Vision (27.6). When FORGE is applied as a plug-and-play aligner on top of Nomic-Vision, the combined system reaches \textbf{33.3} nDCG@10 -- exceeding the best text-only retriever (32.2) -- demonstrating that \textit{query alignment} is the key bottleneck in multimodal-to-text retrieval. https://github.com/mm-bright/multimodal-reasoning-retrieval

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。