IMEX-FND：面向多模态假新闻检测的可追踪交互感知混合专家框架
IMEX-FND: A Traceable Interaction-Aware Mixture-of-Experts Framework for Multimodal Fake News Detection
浏览论文内容
中文总结 AI 辅助
提出IMEX-FND混合专家框架，通过自适应路由与交互分解实现可追踪的多模态假新闻检测，在三个基准上以更少参数超越基线0.4%-1.2%。
中文摘要 AI 辅助
多模态假新闻检测（FND）日益要求裁决不仅准确而且可追踪，以揭示跨模态证据如何被组合，但仍存在两个相互关联的难题。首先，文本-图像关系具有异质性：独特性、冗余性和协同性并存且因帖子而异，因此单一的全局融合规则既脆弱又不透明。其次，主导模态在不同实例间发生转移，静态编码器和固定融合路径难以有效应对。我们提出IMEX-FND，一个交互感知的混合专家框架，将自适应路由与显式交互分解相结合。多模态专家网关（MMEG）在专门专家和共享专家上执行实例级、模态内路由，并构建基于CLIP的跨模态流，生成三个精炼的、可交互的表示，以适应每个帖子的主导模态。随后，交互感知多模态专家融合（IMEF）模块将这些流之间的交互分解为独特性、冗余性和协同性，通过模态替换训练信号生成透明的样本级权重。这两个阶段构成一个先路由后分解的单一流水线，其路由和交互权重均可检查，为错误信息诊断提供实例级和数据集级的可追踪性。在Weibo、Weibo-21和Gossip基准上的大量实验表明，IMEX-FND实现了最先进的性能，以更少的参数超越竞争基线0.4%-1.2%，同时提供优越的可追踪性。
英文摘要
Multimodal fake news detection (FND) increasingly demands verdicts that are not only accurate but traceable, revealing how cross-modal evidence is combined, yet two coupled difficulties remain. First, text-image relations are heterogeneous: uniqueness, redundancy, and synergy coexist and vary from post to post, so a single global fusion rule is brittle and opaque. Second, the dominant modality shifts across instances, which static encoders and a fixed fusion pathway handle poorly. We present IMEX-FND, an interaction-aware mixture-of-experts framework that couples adaptive routing with explicit interaction decomposition. A Multi-Modal Expert Gateway (MMEG) performs instance-wise, within-modality routing over specialized and shared experts and builds a CLIP-grounded cross-modal stream, yielding three refined, interaction-ready representations that adapt to the dominant modality of each post. An Interaction-aware Multi-Modal Expert Fusion (IMEF) module then decomposes the interactions among these streams into uniqueness, redundancy, and synergy, producing transparent, sample-wise weights via a modality-replacement training signal. The two stages form a single route-then-decompose pipeline whose routing and interaction weights are both inspectable, offering instance- and dataset-level traceability for misinformation diagnosis. Extensive experiments on the Weibo, Weibo-21, and Gossip benchmarks show that IMEX-FND achieves state-of-the-art performance, surpassing competitive baselines by 0.4-1.2% while offering superior traceability with fewer parameters.
发表机构
- Sydney Smart Technology College, Northeastern University（悉尼智慧科技学院，东北大学）
- School of Computer and Communication Engineering, Northeastern University at Qinhuangdao（秦皇岛分校计算机与通信工程学院，东北大学）
机构由 AI 辅助整理，请以论文原文为准。
