超越单模态捷径：MLLMs作为跨模态推理器用于 grounded 命名实体识别
Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition
AI总结：
本文提出MCR方法，通过多风格推理模式注入和约束引导的可验证优化，解决MLLMs在跨模态推理中的模态偏见问题，提升 grounded 命名实体识别的性能。
摘要：
grounded 多模态命名实体识别（GMNER）旨在提取文本基于实体，为其分配语义类别，并将其 grounding 到相应的视觉区域。在本工作中，我们探索了多模态大语言模型（MLLMs）以端到端的方式执行GMNER的潜力，超越了它们在级联管道中作为辅助工具的典型角色。关键的是，我们的研究揭示了一个根本性挑战：MLLMs表现出模态偏见，包括视觉偏见和文本偏见，这源于它们倾向于采取单模态捷径，而不是严谨的跨模态验证。为了解决这个问题，我们提出了模态感知一致性推理（MCR），通过多风格推理模式注入（MRSI）和约束引导的可验证优化（CVO）来强制执行结构化的跨模态推理。MRSI将抽象约束转化为可执行的推理链，而CVO使模型能够动态地将其推理轨迹与组相对策略优化（GRPO）对齐。在GMNER和视觉 grounding 任务上的实验表明，MCR有效缓解了模态偏见，并在与现有基线相比时实现了更优越的性能。
英文摘要：
Grounded Multimodal Named Entity Recognition (GMNER) aims to extract text-based entities, assign them semantic categories, and ground them to corresponding visual regions. In this work, we explore the potential of Multimodal Large Language Models (MLLMs) to perform GMNER in an end-to-end manner, moving beyond their typical role as auxiliary tools within cascaded pipelines. Crucially, our investigation reveals a fundamental challenge: MLLMs exhibit $\textbf{modality bias}$, including visual bias and textual bias, which stems from their tendency to take unimodal shortcuts rather than rigorous cross-modal verification. To address this, we propose Modality-aware Consistency Reasoning ($\textbf{MCR}$), which enforces structured cross-modal reasoning through Multi-style Reasoning Schema Injection (MRSI) and Constraint-guided Verifiable Optimization (CVO). MRSI transforms abstract constraints into executable reasoning chains, while CVO empowers the model to dynamically align its reasoning trajectories with Group Relative Policy Optimization (GRPO). Experiments on GMNER and visual grounding tasks demonstrate that MCR effectively mitigates modality bias and achieves superior performance compared to existing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
