超越虚假信号：通过反事实推理与自适应专家路由为多模态大语言模型去偏
Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing
AI总结：
本文提出基于因果中介的多模态大语言模型去偏框架，利用反事实示例区分核心语义与虚假上下文，并结合动态路由的专家混合架构，在讽刺检测和情感分析任务上显著超越现有方法。
摘要：
多模态大语言模型（MLLMs）在整合视觉与文本信息方面已展现出显著能力，但常常依赖虚假相关，削弱了其在复杂多模态推理任务中的鲁棒性与泛化能力。本文通过一种新颖的基于因果中介的去偏框架，应对MLLMs中表层相关偏差这一关键挑战。具体而言，我们借助反事实示例区分核心语义与虚假的文本和视觉上下文，以激活训练阶段的去偏，并采用具有动态路由的专家混合（MoE）架构，选择性地调用模态特定的去偏专家。在多模态讽刺检测与情感分析任务上的实证评估表明，我们的框架显著超越了单模态去偏策略及现有的最先进模型。
英文摘要：
Multimodal Large Language Models (MLLMs) have shown substantial capabilities in integrating visual and textual information, yet frequently rely on spurious correlations, undermining their robustness and generalization in complex multimodal reasoning tasks. This paper addresses the critical challenge of superficial correlation bias in MLLMs through a novel causal mediation-based debiasing framework. Specially, we distinguishing core semantics from spurious textual and visual contexts via counterfactual examples to activate training-stage debiasing and employ a Mixture-of-Experts (MoE) architecture with dynamic routing to selectively engages modality-specific debiasing experts. Empirical evaluation on multimodal sarcasm detection and sentiment analysis tasks demonstrates that our framework significantly surpasses unimodal debiasing strategies and existing state-of-the-art models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
