MMEmb-R1: 基于推理增强的多模态嵌入与配对感知选择及自适应控制
MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control
AI总结：
本文提出MMEmb-R1框架，通过引入配对感知推理选择和强化学习，解决多模态嵌入中推理与对比监督不匹配及推理冗余的问题，实验显示其在MMEB-V2基准上达到71.2分，参数更少且推理效率更高。
摘要：
MLLMs已在多模态嵌入任务中取得成功，但其生成推理能力未被充分利用。直接将链式推理引入嵌入学习引入两个根本性挑战：第一，实例级推理与成对对比监督之间的结构不匹配可能导致捷径行为，即模型仅学习推理的表层格式；第二，推理并非对所有嵌入任务都有益。强制所有输入使用推理可能引入不必要的计算和延迟，甚至在简单情况下掩盖显著的语义信号。为了解决这些问题，我们提出了MMEmb-R1，一个基于自适应推理的多模态嵌入框架。我们将推理作为潜在变量，并引入配对感知推理选择，通过反事实干预识别对查询-目标对齐有益的推理路径。此外，我们采用强化学习来选择性地在必要时调用推理。在MMEB-V2基准上的实验表明，我们的模型仅使用4B参数即可达到71.2分，建立了新的最先进的性能，同时显著减少了推理开销和推理延迟。
英文摘要：
MLLMs have been successfully applied to multimodal embedding tasks, yet their generative reasoning capabilities remain underutilized. Directly incorporating chain-of-thought reasoning into embedding learning introduces two fundamental challenges. First, structural misalignment between instance-level reasoning and pairwise contrastive supervision may lead to shortcut behavior, where the model merely learns the superficial format of reasoning. Second, reasoning is not universally beneficial for embedding tasks. Enforcing reasoning for all inputs may introduce unnecessary computation and latency, and can even obscure salient semantic signals for simple cases. To address these issues, we propose MMEmb-R1, an adaptive reasoning-based multimodal embedding framework. We formulate reasoning as a latent variable and introduce pair-aware reasoning selection that employs counterfactual intervention to identify reasoning paths beneficial for query-target alignment. Furthermore, we adopt reinforcement learning to selectively invoke reasoning only when necessary. Experiments on the MMEB-V2 benchmark demonstrate that our model achieves a score of 71.2 with only 4B parameters, establishing a new state-of-the-art while significantly reducing reasoning overhead and inference latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
